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Every month, 23 million people use CareerBuilder.com to find their next job. 
_ Dell™ PowerEdge™ servers and Microsoft® SQL Server® helped CareerBuilder 
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, Simplify management, and reduce costs. That’s why we’ve sold 
more SQL Server than anyone else, anywhere in the world. 


da m 
atj 


SIMPLIFY YOUR DATABASE AT DELL.COM/SQL 


*Individual results may vary. 


Dell is a trademark of Dell Inc. ©2008 Dell Inc. All rights reserved. 


fmnancapeno 
TECHNOLOGIES 


O snot HO TOOTHOO oona OIK 
ROO 11 100 19090) 10000 toe oe 


O 


eA. 0 
DOOTIOOOT OO O 


Embarcadero Technologies is throwing down 
the gauntlet to all database administrators, 
developers and data modelers. We have 
i compiled a number of common database 
Embarcadero products let you access your databases from a single problems and each task has an estimated time for completion 
application window — so you can manage multiple databases, across — but we challenge you to do them even faster! 
different DBMS platforms. No other products or native tools let you do 


that. So, why waste your time on anything else? To get started, simply download your evaluation software and 


click on the ‘db FEST - Take the Challenge’ icon to check out the 


step-by-step guide. Then simply record the time taken to complete 
Try it for FREE and see for yourself! We are so sure that you can each task, return the form to us and you will automatically receive 


achieve real benefits in real time that we're offering you the chance to a FREE Embarcadero ‘db FEST tour’ t-shirt — and the chance to 
try all of Embarcadero’s products for FREE in a 14 day trial. Just visit win an even bigger prize! 
www.embarcadero.com/challenge to download your copy now! 
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—the SQL Server Magazine Editors We know you love the 


After many sleepless nights, our editors have handed out awards for the SQL Server Mag forums, 
best products of 2008 in 12 categories, including Product of the Year. Find but don’t forget that our 
out who earned the highest honors in our annual Editors’ Best awards. authors and editors blog 
TT regularly, too. Check out 

Awards Recognize the Best of TechEd 2008 Developers the SQL Server Mag blogs 

—the SQL Server Magazine Editors at www.sqlmag.com/blog/. 
For our seventh annual awards issue, our judges traveled to the TechEd —Sheila Molnar, 


Developers conference in Orlando to take a close look at the finalists and senior editor 


award the top development products. The envelope, please! 


Indexing the Data Warehouse 
—Michelle A. Poolet 


Introducing Excel Services for Learn the best way to index the dimensions and fact tables in 

SharePoint your data warehouse to reduce query response times. 

—Paul R. Vaughn 

Excel Services lets businesses share spreadsheets within SQL Server Profiler or 

Microsoft Office SharePoint Server 2007. Learn how to add Server-Side Trace? 

this powerful BI tool to your bag of tricks and see immedi- —Andrew J. Kelly 

ate business benefits. Did you know that using SQL Server Profiler can have an 
impact on the performance of your production servers? 

HierarchyID, Part 2 Check out these test results comparing SQL Server Profiler 


, with a server-side trace 
—Itzik Ben-Gan 


Use the new HIERARCHYID data type to move a subtree 


from one parent location in the hierarchy to another, and to Sharpen Your Basic SQL 
convert a parent-child representation of the hierarchy to a Server Skills 
representation that uses the HIERARCHYID data type. =Pinalkümar Dave 
Try using a self-join when you want to compare values in the 
Digging into SSMS same column. 
—Roman Rehak 
Explore some lesser-known SQL Server Management Studio T-SQL 101, Lesson 6 
features that help you better manage queries, organize —William McEvoy 
scripts and other task resources, and report database and Create a table“the old fashioned way” to better understand 
server statistics. the table and its columns, and use the resulting T-SQL code 


documenting the table’s structure to easily recreate the table. 
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MSSQL Blocks and Orphan Finder 
—Kevin Kline 


Check out two SQL Server tools that help you find locking and blocking problems 
in your SQL Server instances, as well as records that have values in the foreign key 
column of your database tables that don’t exist in the parent table. 
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Videos and Reader Comments Give 

Sqimag.com Readers Online Help 

—Anne Grubb 

Stay tuned for videos from SQL Server experts on ITT V.net, and be on the lookout 
for helpful nuggets of information in online article comments on Sqlmag.com. 
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Visual Studio .NEXT Wish List 

—William Vaughn 

Love Visual Studio 2008? Hardly. The author points out 

where Microsoft needs to fix VS 2008's shortcomings such 

as the inability to access the new SQL Server data types or _ 

ADO.NET features. 9 i J ' 
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XMLSpy - XML Tools 
for Microsoft® Developers 


Altova® XMLSpy® 2008 — the world’s best-selling XML development 
canna environment for modeling, editing, transforming, and debugging XML 
GO Visual Studio technologies — provides numerous tools that help Microsoft developers 
create today’s most advanced XML-based and Web services applications. 
Download a free, 30-day trial at www.altova.com 


Optimized for 


With support for seamless 

¢ Support for very large XML files integration in Visual Studio®, 
e Support for C#, JavaScript, VBScript, & Java in XSLT 1.0/2.0 engines C# code generation, support 
for working with Open XML 
in Office 2007, and more, 
XMLSpy is the must-have 
XML tool for Microsoft 
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e Project generation extended for Visual Studio 2008 
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e Support for Office Open XML 
(OOXML) in Office 2007 documents 


e Editing & validating XML in Word, 
Excel®, and PowerPoint® 2007 files 
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e Processing Office 2007 data with 
XSLT & XQuery 


Download a free, 30-day trial at www.altova.com 
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Migrating to 


SQL Server 2008 


i no secret that SQL Server 2008 has come too 
soon for most SQL Server customers. Many of 
the organizations that I’ve talked with are still in 
the process of deploying SQL Server 2005. For 
those businesses, a few delays in the final SQL 
Server 2008 code release are actually good news. 
However, when SQL Server 2008 is released later 
this year, many of those businesses will have a de- 
cision to make: Should they migrate from SQL 
Server 2000 to SQL Server 2005, or should they 
jump directly to SQL Server 2008? 


From 2000 to 2008 

At this year’s TechEd 2008 IT Professionals, Sheila 
Molnar, lead editor for SOL Server Magazine, and 
I asked Fausto Ibarra, a director of product man- 
agement for SQL Server at Microsoft, this ques- 
tion, as well as a few others. Ibarra explained that 
in the past, Microsoft has advocated migrating 
from SQL Server 2000 to SQL Server 2005 because 
SQL Server 2008 wasn’t ready and the upgrade to 
SQL Server 2005 would let organizations take ad- 
vantage of the immediate business value that SQL 
Server offers. 

However, after SQL Server 2008 is released, 
customers could migrate directly to SQL Server 
2008. Ibarra pointed out that migrating from SQL 
Server 2000 to SQL Server 2008 will be a fully sup- 
ported scenario. Customers who are still running 
SQL Server 2000 will be able to migrate directly to 
SQL Server 2008 without first having to upgrade 
to SQL Server 2005. This is welcome news not only 
for those customers who are considering migrating 
from SQL Server 2000 to SQL Server 2005 in the 
near future, but also for those who won’t be ready 
to migrate from SQL Server 2000 for some time. 


New Features 

SQL Server 2008 is an especially compelling re- 
lease for customers still running SQL Server 2000 
because it includes all the functionality found in 
SQL Server 2005, in addition to many new fea- 
tures. After you’ve run SQL Server 2005 for awhile, 
it’s easy to take for granted some of its impor- 
tant enhancements. SQL Server 2005 introduced 
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support for SQLCLR and database mirroring, not 
to mention new major subsystems such as SQL Michael Otey 
Server Integration Services, SQL Server Report-  (mikeo@sqlmag.com) is technical director 
ing Services, and SQL Server Service Broker. SQL for Windows IT Pro and SQL Server Maga- 
Server 2008 includes an impressive collection of — 2” a4 autor of Microsoft SQL Serer 
E 2008 New Features (Osborne/McGraw-Hill). 
new features as well as a majority of the features 
that were originally planned for SQL Server 2005 
but were cut from that release because of resource 
and/or time constraints. 
Some of the features that SQL Server 2008 
brings to the table are database compression and 
database backup compression; transparent data 
encryption; new date, time, and spatial data types; 
and a new FILESTREAM data type. Other im- 
portant SQL Server 2008 features include the Re- 
source Governor, which manages resource alloca- 
tion for long-running queries, and Policy-based 
Management, which lets you enforce database and 
server standards across all the SQL Server 2008 
systems in your organization. 


Make the Jump 
If you're still running SQL Server 2000, next year 
will be a great time to make the jump to SQL 
Server 2008. Direct migration is fully supported by 
Microsoft, and SQL Server 2008 will offer many 
new features that can bring a lot of value to your 
business. SQL 
InstantDoc ID 99499 | 
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Find and Replace Keywords in Stored Procedures 


Louis Nguyen 


LISTING |: Sample Code That Shows 
How to Use FindAndReplace.sql 


USE adventureworks 
GO 


EXEC FindAndReplace @myfind='[Employee]', 
@myreplace='[vEmployee]', @filepath='C:\Temp\' 
GO 


Print Message Output 


ORE on the WEB 


Download the code at 
InstantDoc IDs_99204, 
99461, and_99466. 


tedious task that many DBAs perform is modi- 

fying stored procedures when an object name 
changes. For example, a project might require that 
stored procedures no longer reference the Employee 
table but the vVEmployee view instead. In SQL Server 
2000, stored procedures are stored in the syscom- 
ments system table, which is difficult to work with. 
Fortunately, in SQL Server 2005, stored procedures are 
stored in the sys.sql_modules system table. 

The sys.sql_modules catalog view exists in each 
database. It has a row for every object, including 
stored procedures. Each stored procedure’s defini- 
tion is stored in the definition column, 
which is of the nvarchar(max) data type. 
I created a stored procedure, FindAnd 
Replace.sql, that uses sys.sql modules to 
identify stored procedures that contain a 
given keyword in their 
definitions. For each 
stored procedure that 
contains the keyword, 
FindAndReplace.sql out- 
puts an alter script and a 
backup script. The alter 
script, if run, will delete 
the existing definition 
and create a new one that 


match found: uspPrintError 
match found: 
match found: 
! MATCH FOUND: 


uspLogError 
uspGetBillofMaterials 
uspGetEmp]oyeeManagers 
uspGetManagerEmp loyees 
uspGetWhereUsedProductID 
uspUpdateEmployeeHireInfo 
uspUpdateEmployeeLogin 
uspUpdateEmp loyeePersonaliInfo 


|! MATCH FOUND: 
match found: 

!! MATCH FOUND: 
!! MATCH FOUND: 
! MATCH FOUND: 


Figure | 


Sample message showing which stored procedures 
contain the keyword 


contains the replacement 

keyword. The backup 

script stores a copy of the 

unmodified definition. 

FindAndReplace.sql 

takes three parameters: 

* @myfind, which 
specifies the keyword to 
search for 


* @myreplace, which is the replacement keyword 
e @filepath, which specifies the folder in which to 
store the alter and backup scripts 


For example, the code in Listing 1 uses FindAnd 
Replace.sql to search for [Employee] in the definition 
column of sys.sql_modules in the AdventureWorks 
database. For each stored procedure that contains 
[Employee], FindAndReplace.sq! will create an alter 
script to replace [Employee] with [vEmployee]. The 
alter and backup scripts will be stored in the Temp 
folder on the C drive. 

After FindAndReplace.sql runs, it outputs a mes- 
sage like that in Figure 1. In this case, the message lets 
you know that five stored procedures contain the key- 
word [Employee]. So, C:\Temp will contain five alter 
scripts and five backup scripts. 

Here’s how I use FindAndReplace.sql in my job: I 
create a dummy database on my development server, copy 
the stored procedures on my production server to the 
development server, and run FindAndReplace.sq! on the 
development server. After it runs, I use the open-source 
WinMerge tool (winmerge.org) to compare the set of 
alter scripts against the set of backup scripts so that I 
can review the changes the alter scripts will make. When 
Tm satisfied that no definitions will be altered incor- 
rectly, I run the alter scripts on my production server. 

You can download FindAndReplace.sql by 
going to www.sqlmag.com, entering 99461 in the 
InstantDoc ID text box, clicking Go, then clicking the 
99461.zip hotlink. This stored procedure works with 
SQL Server 2005 only and requires that xp_cmdshell 
be enabled. The SQL Server 2005 engine must have 
permission to write files to the specified output folder. 

—Louis Nguyen, DBA, Centex Homes 
InstantDoc ID_9946). 


Move User Tables Between File Groups with Ease 


ost DBAs use GUIs such as SQL Server 2005's 
SQL Server Management Studio (SSMS) or 


Eli Leiba 


8 August 2008 


SQL Server 2000’s Enterprise Manager to move tables 
between file groups. However, performing these tasks 
through the GUI is tedious, especially when you need 
to move a large number of them. In addition, using the 
GUI to move tables is prone to human errors, such as 
not moving all the tables that should be moved. 


I developed a stored procedure, sp_MoveClst- 
TablesToFileGroup, that first finds the tables that 
need to be moved, then moves those tables and their 
data. Specifically, sp_MoveClstTablesToFileGroup 
looks for the clustered-indexed user tables you specify 
and moves those user tables from the old file group to 
the specified new file group. This stored procedure is 
especially helpful when you need to move user tables in 
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The CUTTING EDGE ARCHITECT event that will Visual leda 8 of 
IMPACT how you think, innovate, on 
design solutions, and further your career. 


E Join hundreds of your colleagues in San E Find answers to your toughest software 
Francisco, October 2008, for the best software architecture questions. 


architecture conference anywhere. 
y E Meet with IASA community and education 


B Obtain a practical, original approach for system leaders from all over the globe. 


design and development process. 
j p i B Capitalize on valuable products and services from 


E Learn from the best speakers and thinkers in leading companies in the expo hall. 


the software architecture space. 
7 D Unwind and enjoy a great city location with 


B Tap into trends of the software industry in friends and colleagues. 
dynamic keynote sessions. 


B This premiere Architect event is worth your time 
Network with industry experts and authors of and investment! 
the books you read. 
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KEYNOTES 


MONDAY, OCTOBER 6™, 2008 


WHY SOFTWARE SUCKS 

DAVID PLATT 

Users think that today's software sucks. It’s unsafe, it's unreliable, and it's hard to use. These problems are not technical. We've been able to solve them for many 
years, but instead we've gotten a paper clip with eyebrows. Why? Software sucks because developers forget (or never knew) the bedrock principle of software devel- 
opment: KNOW THY USER, FOR HE IS NOT THEE. For example, what do your customers come to you for? Hint: It's not software. For another example, do you think 
your users care about your application? They don't. Never have; never will. They care about accomplishing the task that it does. They don't want to think about you 
or your application at all. It's your job to care about them anyway. We show good and bad examples from commercial software and Web sites: those that under- 
stand and help their users, and those that treat users with contempt. For example, consider the ads for Microsoft Office that show non-upgrading users wearing 
plastic dinosaur heads. Developers fear looking like dinosaurs by not having the latest technology, but ordinary users fear breaking an installation that currently 
works, or having useless junk like dancing paper clips slow down their computers so they need to buy new ones. Your user is not you. We put this nation on wheels 
not by training the entire population as mechanics, but by improving cars so they didn't often need mechanics. The same transition needs to happen to the soft- 
ware industry. This talk provides sound design principles so that your software won't suck. Learn how blindness will improve your vision. 


TUESDAY, OCTOBER 7™, 2008 


SOFTWARE + SERVICES: A PERSPECTIVE 

DAVID CHAPPELL 

The move to service-orientation is well underway, both inside enterprises and on the Internet. What role does traditional software play in a world of on-line serv- 
ices? In particular, how is Microsoft approaching the combination of software plus services? In this keynote presentation, David Chappell examines this important 
area, providing an overview along with a perspective on this emerging combination. 


WEDNESDAY, OCTOBER 8™, 2008 


DATA ARCHITECTURE AND BEYOND: AGILE APPROACHES FOR ADDRESSING DATA QUALITY CHALLENGES 

SCOTT AMBLER 

Data has always been a critical architectural aspect of business systems and of enterprise infrastructures in general. Yet it is estimated that data quality 
challenges lead to a $600 billion productivity loss annually, a clear indication that traditional approaches to data architecture, data management, and data 
governance have failed to live up to their promises. Without a solid data foundation in place there is little hope that modern, services-based architectures 
will succeed. This keynote explores the technical and cultural challenges surrounding data-oriented activities, compares and contrasts potential strategies 
for addressing data quality problems, and in particular focuses on agile strategies and techniques for doing so. 


POST-CONFERENCE WORKSHOPS The cost of 
a workshop is 
SCHEDULE AT A GLANCE in addition to 
WEDNESDAY, OCTOBER 8", 2008 ¢ 1:30 PM - 5:30 PM the regular 
MONDAY OCTOBER 6.2008 BECOMING AN ARCHITECT: A GUIDED TOUR ee 
8:30am - 9:30am Keynote: David Chappell PAUL PREISS 
10:15am - 1:00pm Conference Sessions This workshop will provide you with the basic structure for becoming an architect. We will cover each 
E i f the fundamental categories according to the most comprehensive education plan in the world 
1:00pm - 2:30 p 
i pm Lunch/ Expo Hall today. The IASA education program includes courses for all key skills an architect needs for success. 
2:30pm - 5:30pm Conference Sessions During this workshop we will cover each category and skill in detail and provide significant examples 
5:30pm - 7:00pm Expo Hall/ Reception of each. By the time you leave this workshop, not only will you know the skills you need, you will be 


TUESDAY, OCTOBER 7, 2008 on your way to acquiring them. 


8:30am - 9:30am Keynote: David Platt EVERY CLASS AS A SERVICE 
10:15am - 1:00pm Conference Sessions JUVAL LOWY 
1:00pm - 2:30pm —Lunch/ Expo Hall WCF is more than just the next generation platform for building connected systems. In many respects, 
; WCF is the next development platform for Windows applications, providing system features that are 
2:30pm - 5:30pm Conference Sessions presently crafted by hand on top of .NET and Windows. With WCF, every class automatically benefits from 
WEDNESDAY, OCTOBER 8, 2008 security, instrumentation, call timeout, error masking, fault isolation, reliability, remote calls, tracing and 
8:00am - 9:00am Keynote: Scott Amber logging, calls buffering, synchronization, interoperability, and with little or no change could also benefit 
from queued calls, transactions management, and various instantiation modes. To maximize the use of 
these off-the-shelf plumbing aspects you should push the service boundary down into your system, but 
taken to its ultimate conclusion—should every class be a WCF services? And what about performance? 
1:30pm - 5:30pm Post Conference Workshops This workshop starts with demonstrating the power and productivity of WCF as a ‘Better .NET’, contrast- 
ing WCF used granularly on every class with classic NET in terms of performance, throughput and scala- 
SEE WEB SITE FOR THE LATEST UPDATES. bility, and will substantiate the provocative claim that every class can and should be a service. 
www.lASAConnections.com 


10:00am- 12:15pm Conference Sessions 
Regular Conference Ends 


October 6-8, 2008 @ San Francisco, CA @3 


IASA SESSIONS all 


IASA01: ANTI-PATTERNS IN SOFTWARE 


PROJECTS: THE HUMAN FACTOR 

ROB DAIGNEAU 

The creation of software products is a highly com- 
plex endeavor. One methodology after the next is 
touted as being the solution to smoother project 
execution, but the pundits never seem to account 
for the chasm that exists between academia and 
reality in the corporate world. This chasm exists 
because of the human factor-the ingredient which 
ultimately has the greatest influence upon the suc- 
cess of any software project. Join us in this session 
to see how people, teams, leadership styles, and pol- 
itics can subvert the benefits that we should be real- 
izing from strategic architecture plans, modern 
methodologies like Agile, and IT governance 
processes. We will identify some of the common 
anti-patterns attributable to people and organiza- 
tions, and look at ways to address these self-defeat- 
ing behaviors in a positive manner. 


IASA02: SERVICE ASPECTS—ASPECT 
ORIENTATED DESIGNS IN DISTRIBUTED 
ENTERPRISE ARCHITECTURE 

ADNAN MASOOD 

Aspect Oriented Programming (AOP) and Aspect- 
Oriented Software Development (AOSD) support the 
software development paradigm which leverages sep- 
aration of concerns, especially cross-cutting concerns 
as a next step to modularization. Separation of con- 
cerns can be defined as breaking down a program into 
distinct parts that overlap in functionality as little as 
possible. The similar concerns are factored and 
defined as aspects which are separated out from the 
main logic making the implementation more main- 
tainable. In this session, we approach the service ori- 
entation as an aspect of a distributed system. Using 
attribute-oriented design for aspect implementation, 
this presentation focuses on merits of exposing serv- 
ice end points from business objects by using AOP 
practices. The attendees will: gather the understand- 
ing of AOP, a fast growing research and development 
area in modern software development; understand 
the state of affairs of AOP in the current IDE's and pro- 
gramming languages especially with Spring, AspectJ 
and Aspect.NET; explore the rationale of aspect-based 
nature of services and deep dive into the open source 
ServiceAspect CodePlex project for a sample imple- 
mentation. This session's focus is the architecture and 
design practices that AOSD brings to the enterprise 
architecture. Best practices and design patterns fol- 
lowed in AOP will be discussed with a demo of 
Aspect.NET and ServiceAspect, which is used to pub- 
lish business objects as WCF services using attributes. 


IASA03: TRACKING, MONITORING, 
METERING AND MEASURING YOUR SOA 
ANANT KADYALA 

SOA has evolved from pilot projects to mission-criti- 
cal initiatives. Today, most companies understand 
the benefits of SOA, but are grappling with the chal- 
lenges involved with the execution. In this session, 
we delve into the metrics that are crucial to keep 
your SOA initiative healthy and on the right track. We 


will discuss a range of technical, people and process 
metrics that need to be closely monitored and man- 
aged closely. We will also discuss tools and best 
practices that can be leveraged to manage your SOA. 


IASA04: ENTERPRISE SERVICE BUS: 

AN OPERATING ENVIRONMENT FOR 

YOUR SERVICES 

CURT PETERSON 

With the rise of service-oriented architecture, enter- 
prise service bus (ESB) technologies have emerged 
as an operating environment for services, allowing 
services to intercommunicate. ESB technologies, 
properly architected and deployed, ensure easy 
scaling and management of services in a service-ori- 
ented environment. In this session we will examine 
the path we have taken to arrive at current-state 
technologies, and we will look at the capabilities 
that contemporary ESBs add to an architect's arse- 
nal. By the end of this session you will have a clear 
understanding of what and ESB is, and why ESBs are 
a solid foundational choice for architects. 


IASAO5: BREAKING THE SUPER 

PLATFORM BONDS 

CHRIS HADDAD 

Proprietary programming interfaces, ‘free’ capabili- 
ties, 24/7 support, comprehensive infrastructure 
suites, and integrated tooling provide application 
teams with compelling reasons to tightly couple appli- 
cations and services to super platform offerings (e.g 
IBM WebSphere, Microsoft .NET/Visual Studio, Oracle 
Fusion Middleware, BEA AquaLogic, and SAP 
Enterprise SOA). While super platform infrastructure 
components are the default choice for projects, the 
platform choice often results in high licensing fees, 
steep learning curve, capability compromises, and 
heavy-weight production deployments. Enterprise 
acceptance of cloud-hosted infrastructure services, 
open source, rebel frameworks, and Software as a 
Service threaten super platform dominance and are 
tipping the balance of power away from incumbent 
vendors. This session explores the forces breaking the 
super platform bonds including: cloud computing and 
infrastructure services; Software as a Service (SaaS) 
and open source business models; emerging platform 
vendors (e.g. SalesForce.com, Google, Amazon); spe- 
cialty vendors; rebel frameworks; and virtualization. 


IASA06: SOA REPORT CARD 

CHRIS HADDAD 

Everyone seems to be doing SOA, but how many 
organizations are doing it well? Is anyone making a 
passing grade? Burton Group has been conducting 
intensive research into real-world SOA initiatives. 
The research is focused on SOA planning and execu- 
tion. It compares and contrasts top-down and bot- 
tom-up approaches. It explores organizational and 
cultural impediments. It examines governance 
strategies. It also looks at business models and 
metrics. We'll present our findings from this 
research in this session. Learn what works and what 
doesn't: Where do you start?; How do you identify, 
model, and describe services?; Is an ESB a prerequi- 


site?; What about WS-* versus REST?; When do you 
really need to establish governance?; How much 
governance is required?; What changes are required 
to the organization, funding models, development 
practices, etc?; How do you measure success? 


IASAO7: HOW TO MANAGE AND 

CONTROL VM SELF-PROVISIONING 

RUSS GIBFRIED 

In the corporate environment, delegating VM self- 
provisioning tasks to your user community allows 
your customers to quickly setup, build, and tear down 
complex solutions on your host's datacenter, with lit- 
tle knowledge or concern about the hardware and 
network infrastructure required. For architects, VM 
self-provisioning promises users a more flexible envi- 
ronment where they can rapidly respond to business 
needs. But how do architects and administrators con- 
trol their virtual infrastructure so they can provide 
the flexibility and attractiveness of the model, and 
still meet enterprise requirements for security, sta- 
bility, and reduced costs? This session provides real- 
life examples using Microsoft System Center prod- 
ucts Virtual Machine Manager 2007, Operations 
Manager 2007, and Configuration Manager 2007 to 
provide SAIC line organizations a secure virtual envi- 
ronment to develop high impact solutions. 


IASAO8: ARCHITECTURAL STRATEGIES, 
PATTERNS, AND FORCES FOR HIGH- 
VOLUME WEB SITES 

RANDY SHOUP 

Websites, services, and enterprises are increasingly 
faced with growing demands-more users, more 
traffic, more data. Learning principles and patterns 
for scaling helps architects and developers in those 
organizations meet and exceed those demands. 
This session covers the fundamental architectural 
principles eBay has followed to grow and evolve its 
infrastructure to a massive scale. It covers the 
forces (or "-ilities") architects need to contend with 
and design for—scalability, availability, manageabil- 
ity, etc.—and outlines eBay's core set of architectur- 
al strategies that meet—and trade off—those forces. 
This session offers reusable patterns for each strat- 
egy, and provides specific examples from the eBay 
infrastructure to illustrate the patterns in action. 


IASAO9: THE BUSY ARCHITECT'S GUIDE TO 
RULES AND RULES ENGINES 

TED NEWARD 

If you've been keeping your ear to the ground, you 
may have heard some talk recently about “rules”, 
“business rules” and “rules engines”, but not neces- 
sarily any clear discussion on what they are, how to 
use or design them, or why they might be useful or 
important. This presentation puts some concrete 
definition around what a “rule” is, how a "rule 
engine” like Drools (a.k.a JBoss Rules), JESS or the 
rules engine found in Windows Workflow can enable 
your users to be more agile than they ever thought 
possible. We'll also talk about where it fits into both 
the WCF/WF/BizTalk, J2EE, Spring and “lightweight” 
development environments, and how you can (final- 
ly!) get out of the "infinite if-else game.” 
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IASA10: RETHINKING ENTERPRISE 

TED NEWARD 

The era of the big, heavy, transactionally-oriented 
client/server topology, as best described by the 
J2EE Specification and Blueprints document, seems 
to be over. The era of the lightweight, transactional- 
ly-oriented client/server topology seems to be at its 
zenith. Is this really where we want to be when 
building enterprise systems? And how did we end 
up here, anyway? What's the new “enterprise” 
developer supposed to do? 


IASA11: PRAGMATIC ARCHITECTURE 

TED NEWARD 

Building an application is not the straightforward exer- 
cise it used to be. Decisions regarding which program- 
ming languages to use (Java, .NET, even FoxPro), which 
architectural approaches to take (n-tier, client/server), 
which user interface approaches to take (Smart/rich 
client, thin client, Ajax), even how to communicate 
between processes (Web services, distributed objects, 
REST)... it's enough to drive the most dedicated design- 
er nuts. This session discusses the goals of an applica- 
tion architecture and why developers should concern 
themselves with architecture in the first place. Then it 
dives into the meat of the various architectural consid- 
erations available; the pros and cons of JavaWebStart, 
ClickOnce, Windows Presentation Foundation, SWT, 
Swing, WinForms, Struts, WebForms, Ajax, RMI, .NET 
Remoting, JAX-WS, ASMX, Windows Communication 
Foundation, Windows Workflow Foundation, JMS, 
MSMQ, transactional processing, and more. After that, 
the basic architectural discussion from the first part is, 
with the aid of the audience in a more interactive 
workshop style, applied to a real-world problem, dis- 
cussing the performance and scalability ramifications 
of the various communication options, user interface 
options, and more. 


IASA12: MANAGING, SELECTING AND 
SUPPORTING QUALITY ATTRIBUTES IN THE 
REAL WORLD ENTERPRISE ARCHITECTURE 
ENVIRONMENT 

ANGELA YOECHEM 

Quality Attributes are the foundational “-ilities" we 
build into every technology solution. Making the 
appropriate optimization decision for one quality 
attribute over another is a particularly esoteric 
aspect of architecture engagement. In this presen- 
tation, we'll provide guidelines for determining 
which attributes should be encouraged and 
tracked—and when-in your enterprise. 


IASA13: "DESIGN PATTERNS” 

IN DYNAMIC LANGUAGES 

NEAL FORD 

The Gang of Four book was actually 2 books: a 
nomenclature describing common software prob- 
lems and a recipe book for solutions. The vocabulary 
they defined is still useful. The recipes are a disaster! 
Dynamic languages (like Groovy and Ruby) have pow- 
erful meta-programming facilities far beyond stati- 
cally typed languages. It turns out that many of the 
structural design patterns in the Gang of Four book 


and beyond are much easier to solve with meta-pro- 
gramming. This session compares and contrasts the 
“traditional” approach of design patterns with a 
more nuanced meta-programming approach. Using 
language features creates cleaner abstractions with 
fewer lines of code and little or no additional struc- 
ture. This session shows one of the many reasons 
that dynamic languages are such a hot topic. 


IASA14: EVOLUTIONARY SOA 

NEAL FORD 

Managers and ivory tower architects seem to think 
that all the rules that apply to "normal" software 
don't apply to SOA. Ironically, they matter even more. 
Agility and SOA are closely aligned because SOA is 
about building complex distributed systems and 
Agility is about effectively building complex software. 
This session unveils the pillars of successful SOA and 
how to achieve them in a testable, iterative fashion. It 
discusses testing strategies, how to make your archi- 
tecture more robust and maintainable, and how to 
design an evolutionary architecture. 


IASA15: TEST-DRIVEN SOA DEVELOPMENT 
FRANK COHEN 

Software architects and developers have adopted 
"test first" software development strategies widely. 
Unit test frameworks for Java, .NET, Web services, Ajax, 
REST, and SOA show the diversity of adoption. In this 
session, Frank Cohen will identify a new methodology 
to repurpose unit tests as functional tests to conduct 
acceptance and regression testing. Cohen will show 
how to repurpose these tests without any extra effort 
as load and performance tests to understand the 
Scalability Index of a service for hardware, software, 
and resource forecasting. Cohen will then show repur- 
posing these tests as business service monitors to 
prove Service Level Agreement (SLA) compliance and 
then show how you accomplish this using open-source 
and commercial test tool platforms. 


IASA16: SECURITY 101: AN INTRODUCTION 
TO SOFTWARE SECURITY AT THE 
ARCHITECTURAL AND PROCESS LEVEL 
ALLEN HOLUB 

As more and more of our applications move onto 
the Web, security becomes even more critical. Good 
security, however, has to be built in, not tacked on 
as an afterthought. Misconceptions about security— 
that an application can be made secure solely by 
using encryption (https) and firewalls, for example— 
bound. Moreover, security is more an architectural 
(at both the system and program level) and process 
problem than a coding problem. This session gives 
you an overview of what it means to make an appli- 
cation truly secure. We'll start by looking at a cou- 
ple common attack scenarios so that you can 
understand the sorts of things that make an appli- 
cation vulnerable, then we'll talk about how to 
design and build your applications so that they will 
be truly secure. Covered topics include security 
architectures, code and design review, penetration 
testing, risk analysis and risk-based testing, securi- 
ty-related requirements, static analysis, abuse 
cases, security operations, and crypto. 


IASA17: THE OBJECT-ORIENTED-DESIGN 
PROCESS AND UML 

ALLEN HOLUB 

Good 00 designs are typically the result of an order- 
ly process: problem-statement definition, use-case 
analysis, dynamic modeling, and class design (typi- 
cally in that order). Every stage of that process 
involves artifacts, both in English and in UML. UML, 
however, is just a notation—a way to make your 
design ideas clear on paper. Many people learn the 
notation, however, without any idea of how that 
notation integrates with the design process. Even 
worse, many people see UML as, somehow, a process 
of its own. This session provides a succinct overview 
of the 00-Design process, showing you how UML 
(and other artifacts, such as the user-interface 
design) integrate into that process. It's a big-picture 
class, showing you how the entire design process 
works, and how the various parts of the process 
interact with one another, rather than focusing on 
one aspect of that process (like modeling). 


IASA18: USING JAVA MESSAGE SERVICE AS 
THE "GLUE" IN HETEROGENEOUS 
ENTERPRISE ARCHITECTURES 

WAYNE CITRIN 

Enterprise architectures based exclusively on Java 
Enterprise Edition (JEE) technology have long relied 
on Java Message Service (JMS) as the communica- 
tions “glue” that holds together the disparate compo- 
nents. But, as other non-JEE technologies like NET are 
added to the mix, how can these new components 
communicate with the existing infrastructure? The 
problem becomes even more severe when the exist- 
ing components can't be modified. This session will 
discuss how JMS can continue to be used as a mes- 
saging infrastructure even when components are 
based on platforms other than JEE. We will start with 
a brief discussion of JMS fundamentals, followed by a 
discussion of the use of JMS over Web services, and of 
various cross-platform approaches using native JMS 
wire protocols. The advantages and disadvantages of 
each approach will be discussed, and guidelines and 
best practices will be presented. Each approach dis- 
cussed will be accompanied by a demo. 


IASA19: YOU'RE NOT AN ARCHITECT, BUT 


NEITHER AM | 

PAUL PREISS 

In this session we will delve into the nature of the 
architecture profession. What is the difference 
between a developer, engineer and software archi- 
tect? Does an architect think differently? What 
exactly are their skills? Is there a difference 
between architecture and design? How does one 
become an architect? How many different opinions 
and perspectives on the role exist in the industry 
(you will be surprised by this)? Most importantly, we 
will discuss the two years of discussions, research, 
opinions and development by IASA members on 
what it really takes to be a software, infrastructure 
or business architect. 
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ARCO1: THE HIDDEN ARCHITECT—BUILDING 
SOLUTIONS FOR SMALLER ORGANIZATIONS 
DAN APPLEMAN 

Large enterprises and governments with large proj- 
ects can justify and afford formal system architecture 
and design, and can afford the loss when a project 
self-destructs (as they so often do). But what about 
smaller businesses and their web applications? These 
organizations fall victim daily to poorly designed 
solutions that aren't what they needed in the first 
place. They need help figuring out what they really 
need—help they often don't realize they need and are 
reluctant to pay for. It's not enough to be a web or 
application developer when it comes to this kind of 
client or project—you have to be a system architect as 
well, even though you don't wear the title and may not 
be able to charge for the service. So how do you 
become a system architect? This session might be a 
good start. We'll cover requirements, economics, 
client management, creative and flexible design 
strategies, and how to look beyond “best practices” 
design patterns to find solutions that fit the client, 
instead of trying to fit the client to the solution. PS. 
Bring your favorite horror stories for show and tell! 


ARCO2: SOAP/WS-* AND REST: 
COMPLEMENTARY COMMUNICATION STYLES 
DAVID CHAPPELL 

Two approaches to creating Web services are most vis- 
ible today. One, using SOAP and the WS-* specifica- 
tions, follows in the footsteps of earlier distributed 
computing technologies. The other, the RESTful style, 
is explicitly based on the principles of the Web itself. 
Both have value, and going forward, both will certain- 
ly be used. This session describes these two approach- 
es, looks at when each one makes sense, and shows 
how Windows Communication Foundation (WCF) can 
support applications built using either style. 


ARCO3: DOMAIN-DRIVEN DESIGN-PART 1 
DINO ESPOSITO 

Business logic represents what the application needs 
to do for the domain you're working in. Business logic 
includes input-based calculation, validation, and dis- 
patch of commands to data sources. Ideally, the busi- 
ness logic completely hides the data sources and 
reports to the presentation layer. How would you 
organize such logic? What's the best and most com- 
monly used approach? And what are the alternate 
solutions? In this session, you'll get up close and per- 
sonal with common-use patterns for modeling busi- 
ness logic and understand their relationship to exist- 
ing software technologies such as table adapters and 
LINQ to SQL. You'll see pros and cons of the 
Transaction Script pattern and understand the rea- 
sons that made the Table Module pattern so success- 
ful in .NET applications. Finally, the session will cover 
the pillars of the Domain Model pattern, the most 
sophisticated and powerful pattern for business logic. 


ARCO4: DOMAIN-DRIVEN DESIGN—PART 2 
DINO ESPOSITO 

Domain-driven design (DDD) is a possible approach 
to developing software in the context of complex 
(and typically enterprise) scenarios. DDD is not for 
just any software project, even though its principles 
sound usually attractive to almost any developers 
and architects regardless of the context in which 
each operate. The primary goal of DDD is accelerat- 
ing the development of software in projects that 
have to deal with quite complex domains. DDD 
focuses on the problem's domain and related logic 
and requires that you base your design on a data 
model. The data model comes out as completely 
decoupled from its persistence layer which raises 
the need for an OR/M tool to serialize and deserial- 
ize the model to and from a data store. In this ses- 
sion, you'll understand the motivation for DDD and 
explore patterns that provide guidance on how to 
effectively design the model for your scenario. 


ARCO5: SOFTWARE ARCHITECTURE FOR 
REALTIME SYSTEMS 

DR. NEIL ROODYN 

Does your software performance suck? Does it take 
over a second for your application to load? Come to 
his session and find out how to improve perform- 
ance. Software architectures often overlook one of 
he most important issues in software today, perform- 
ance. In both soft real-time and hard real-time sys- 
ems there are software architectures that can be 
applied to make the systems you build behave in a 
imely manner. In this session, Dr. Neil (who holds a 
PhD in Software Architecture for Distributed Real- 
Time Systems) will discuss the common issues with 
building software that has high performance patterns 
hat can be worked towards in order to increase the 
performance of your software. 


ARCO6: IT'S ALL WRONG! CHANGE IT ALL! 
DR. NEIL ROODYN 

Sometimes we make mistakes. Yes it's true and if you 
are mature enough to admit that you don't always get 
it right first time then this session will help you. In this 
session, you'll learn a set of recovery practices that 
can help you radically change the underlying archi- 
tecture of your software without impacting the func- 
tionality you have already built. This session is about 
much more than just refactoring. Dr. Neil will discuss 
the pain of making these changes in terms of the 
impact to the egos of the developers that built the 
software, the clients that are paying for the software, 
and the people managing the project and leading the 
development team. Change can hurt; consider this 
session your first step in the recovery process. 


ARCO7: ARCHITECTING CUSTOM TEST 
AUTOMATION WITH VISUAL STUDIO 

TEAM SYSTEM 

JAMES MCCAFFREY 

Writing lightweight custom test automation harness 
with languages such as C# and Perl has several 
advantages over using commercial or open source 
test frameworks. However, the primary disadvantage 
of custom test automation is manageability; your 
test effort can be overwhelmed by the sheer volume 
of test case data files, test scripts, and test result 
files. Visual Studio Team System provides you with 
the ability to design and manage a test effort which 
includes custom test automation. Topics covered 
include: understanding test architecture, creating a 
Generic test type, managing custom test automation, 
emitting detailed .trx test results, and managing test 
result storage. You will leave this session having a 
firm grasp of how and when to use Visual Studio 
Team System to manage custom test automation. 


ARCO8: SERVICE-ORIENTATION: WHERE WE 
ARE AND HOW DID WE GET HERE? 

JUVAL LOWY 

Contrary to common wisdom, service-orientation is 
not just for high-end applications. Every application 
should be service-oriented. But what is service-orien- 
tation really about? What does it mean for mere 
developers? Is there substance behind the hype? In 
this conceptual session, Juval will demystify service- 
orientation for you, examine the history of software 
engineering, and introduce the basic motivation for 
service-oriented applications and their operating 
principal and concepts. 


ARCO9: SERVICE-ORIENTED 

DEVELOPMENT PROCESS 

JUVAL LOWY 

When you develop a service-oriented application, it 
would be naive of you to expect that the only things 
you will do differently will be limited to design and 
technology. The development process itself needs to 
be service-oriented. You cannot “stare into the fire” 
without a mature service-oriented development 
process supporting your effort. This session presents 
you with a service-oriented development process that 
you can apply to your SOA-based products to achieve 
robust applications, manage requirements, plan and 
track your progress, and ensure faster time to market. 


ARC10: CODE GENERATION IN THE 
ENTERPRISE 

KATHLEEN DOLLARD 

Code generation offers one of the most viable options 
for directly expressing the details of your architecture 
throughout your enterprise. You'll retain confidence 
that the architecture is being used correctly, limit or 
generate certain types of tests, and allow evolution of 
the architecture under your control. This session cov- 
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ers an introduction into template styles and then 
moves on to focus on the big picture of code genera- 
tion in a tool-neutral manner. You'll see approaches to 
extracting metadata stores, managing template sets, 
providing enterprise configuration, protecting hand- 
crafted code and versioning templates. Whether 
you're new to code generation or ready to improve its 
effectiveness in your enterprise, you'll leave this ses- 
sion with new perspectives and new techniques. 


ARC11: BLOCK BASED DEVELOPMENT 
KATHLEEN DOLLARD 

Applications run today in chunks. They reside in thin 
and thick layers on intelligent devices, web servers, 
and application servers. We chop up applications 
across logical data tiers, physical location, and owner- 
ship. This session focuses on the human scale engi- 
neering behind these chunks and the tools availabl 
to manage these blocks in the broader picture of you 
application—comparing and contrasting services 
workflow activities, and plug-ins/add-ins. You'll see 
how these differing approaches are synergistic as this 
session spends a lot of time with how to leverage 
them working together—including add-ins supporting 
services, workflows calling services, and services call- 
ing workflows. You'll leave this session better pre- 
pared to choose the right tools to build application 
ecosystems developed and maintained in manage- 
able blocks that can be individually prioritized and 
completed by reasonably-sized teams in response to 
business requirements. 


ARC12: BRIDGING AGILE AND FORMAL 
DEVELOPMENT METHODOLOGIES 

KEN SPENCER 

Many organizations today are choosing to use some 
form of Agile process such as Scrum or MS Agile for 
development. But many of these same teams are 
faced with more and more formal oversight. This 
session drives into the real-world aspects of bridg- 
ing these two worlds. We will discuss lessons learned 
from major project failures that were resurrected 
with a blend of both Agile and formal processes. 


ARC13: SOLVING THE 
BUILD/DEPLOY/INSTALL PROBLEM AND 
SMOOTHING OUT APPLICATION TEST 
CYCLES 

KEN SPENCER 

Most large projects fail for a variety of reasons. One of 
the key ones is the lack of a reliable automated build 
/ deploy / install system. This session will discuss how 
to solve this problem using MSBuild, TeamBuild and 
other Microsoft technologies, and discuss how other 
3rd-party products can fit as well. The session is built 
from lessons learned on two major projects with large 
development teams. 


a o 


ARC14: FEDERATED SECURITY 
IMPLEMENTATION PATTERNS 

MICHELE LEROUX BUSTAMANTE 
Federated security models are a fast-growing necessi- 
ty for enterprise systems to normalize application 
authentication mechanisms, to provide more granular 
control over the security model, and to support single 
sign-on (SSO) to applications and services. The partic- 
ipant and protocols involved in a federated security 
model are consistent across technology platforms 
with Web services standards such as WS-Trust, WS- 
Federation, and SAML living at the heart of it. This ses- 
sion will explore the participants in a typical federated 
security model and illustrate several authentication 
and authorization implementation patterns that 
impact how security tokens are generated, the claims 
they carry, and the way that applications and services 
authorize access. Available platforms for implement- 
ing federated security models will also be discussed. 


ARC15: ROUTING PATTERNS FOR YOUR SOA 
MICHELE LEROUX BUSTAMANTE 
Services deployed within a SOA environment often 
require a routing service to provide necessary securi- 
ty boundaries within the architecture; to provide 
asynchronous logging and message tracking services; 
for capacity planning or priority routing; for content- 
based routing and more. This session will illustrate 
relevant routing architectures; discuss semantics for 
mixing transports between client, router, and service; 
explain the semantics of SOAP addressing that are rel- 
evant to the routing process; discuss the differences 
between pass-through routers and processing routers 
in how they handle messaging protocols; and explain 
how security can be handled between client, router, 
and service. 


ARC16: ARCHITECTING, DESIGNING AND 
IMPLEMENTING YOUR DATA ACCESS LAYER 
DINO ESPOSITO 

For years, architects designed applications modeling 
the domain of the problem through objects. And, for 
years, architects dealt with the conceptual and tech- 
nical difficulties of persisting an object model to a 
relational DBMS. A generally agreed abstract model 
exists for the data access layer of a system that 
includes a persistence layer and a service layer for 
scripting the domain object model. Inside this overall 
model, though, a number of equally valid approaches 
may be taken that result in even significantly differ- 
ent architectures and may require different imple- 
mentation technologies. In this session, we'll first 
review the general scheme of a data access layer, and 
then drill down into options and technologies for the 
persistence and service layers. Along the way, we'll 
touch on concrete technologies such as LINQ to SQL, 
LINQ to Entities, products such as commercial 0/RMs, 
and patterns such as Active Record, Data Mapper, and 
Unit-of-Work. 


ARC17: ARCHITECTS: 

HOW DO THEY HAPPEN? 

ROCKFORD LHOTKA 

In a company where | worked we lost an employee 
with six months total experience because we wouldn't 
promote him to “architect”. Another company hired 
him in that role a week later. Is it possible to be an 
architect with six months experience? Six years? What 
is it that makes some people valuable architects after 
just a few years, while others never even come close? 
Is it possible to cultivate architects? Train them? Or do 
they just spontaneously “get it"? Nearly every organ- 
ization is crying out for quality architects at the 
enterprise, application and systems levels. Come join 
in this nature vs. nurture debate. Learn what person- 
al attributes you should look for or cultivate in 
prospective architects. Learn some techniques that 
do (and don't) work, and information that will directly 
apply to your organization as you try to hire or create 
your own architects. 


ARC18: ACHIEVING BALANCE 

ROCKFORD LHOTKA 

It is all too easy to become fixated on the latest trend. 
Even if we know there's no “silver bullet" it is so 
tempting to become “pure 00" or “pure SOA” or “pure 
anything.” Of course, reality is messier. Real applica- 
tions require a mix of technologies and concepts, 
including SOA, messaging, 00, client/server, workflow, 
and more. At the same time, it isn't chaos. Each tech- 
nology fits perfectly in some cases, acceptably in oth- 
ers, and must be coerced into still others. You can 
achieve balance by using the right technology to solve 
the right problems. The key is to start with a flexible 
architectural philosophy, and to fit technologies into 
that technology without being overly affected by mar- 
keting hype or developer enthusiasm for the latest 
fad. Learn about one such architectural philosophy 
and how it can be used to frame the use of the many 
Microsoft technology offerings available today. 


ARC19: SERVICE-ORIENTED WORKFLOW 
WITH WCF AND WF 

RON JACOBS 

.NET 3.5 brings together two powerful technologies 
with WCF and WF for the first time. This combination 
provides an array of interesting design decisions. 
Should every service be a workflow? How does the 
interaction of the workflow system and WCF impact 
the throughput and scalability of the overall sys- 
tem? When is it better to not combine these two 
technologies and how would you decide on this? 
These are just a few of the questions we will be 
answering in this session. 
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SCOTT AMBLER IBM 
| Scott W. Ambler is the Practice Leader Agile Development 
| with IBM Rational and he works with IBM customers 
around the world to improve their software processes. He 
is the founder of the Agile Modeling (AM), Agile Data (AD), 
Agile Unified Process (AUP), and Enterprise Unified 
Process (EUP) methodologies. Scott is the (co-author of 19 
books, including Refactoring Databases, Agile Modeling, Agile Database 
Techniques, The Object Primer 3rd Edition, and The Enterprise Unified 
Process. Scott is a senior contributing editor with Dr. Dobb’s Journal. His 
personal home page is www.ibm.com/rational/bios/ambler.html and his Agile 
at Scale blog is www.ibm.com/developerworks/blogs/page/ambler. 


DAN APPLEMAN Desaware, Inc. 


Daniel Appleman is the president of Desaware Inc., a 

developer of add-on products and components for 

Microsoft Visual Studio, and is a cofounder of Apress pub- 

lishing. He is the author of numerous books including, 

Moving to VB .NET: Strategies, Concepts and Code, 
Always Use Protection: A Teens Guide to Safe 
Computing and Dan Appleman's Visual Basic Programmer's Guide to the 
Win32 API. He is also the author of a series of e-books on .NET-related topics 
including Regular Expressions with .NET, Visual Basic .NET or C#: Which 
to Choose, Tracing and Logging with .NET and more. 


DAVID CHAPPELL Chappell & Associates 

David Chappell is Principal of Chappell & Associates in 

San Francisco, California. Through his speaking, writing, 

and consulting, he helps IT professionals around the 

world understand, use, and make better decisions about 

enterprise software. David has been the keynote speaker 
for dozens of conferences and events in the U.S., Europe, 
Asia, and Latin America. His popular seminars have been attended by tens of 
thousands of developers, architects, and decision makers in forty countries. 
He has also spoken at many universities, including the National University of 
Singapore, Moscow State University, and Sweden's Uppsala University. David's 
books have been published in ten languages and used regularly in courses at 
MIT, ETH Zurich, and other educational institutions. 
He is Series Editor for Addison-Wesley's award-winning Independent 
Technology Guides, and he's been a columnist for several publications. In his 
consulting practice, David has helped clients such as Hewlett-Packard, IBM, 
Microsoft, Stanford University, and Target Corporation adopt new technolo- 
gies, market new products, train their sales staffs, and create business plans. 
Earlier in his career, he wrote software for supercomputers, chaired a U.S. 
national standardization working group, and played keyboards with the 
Peabody-award-winning Children's Radio Theater. David holds a B.S. in 
Economics and an M.S. in Computer Science, both from the University of 
Wisconsin-Madison. 


WAYNE CITRIN JNBridge 

Wayne Citrin is the Chief Technology Officer at NBridge, 

the Java/.NET interoperability company. He is the archi- 

tect of JNBridgePro, and has been engrossed in Java and 

.NET interop issues since .NET's beta days, over six years 

ago. Previously, Wayne was a leading researcher in pro- 
gramming languages and compilers, and was on the 
Computer Engineering faculty at the University of Colorado, Boulder. He was 
a researcher at IBM's research lab in Zürich, Switzerland, and has a PhD from 
the University of California, Berkeley, in Computer Science. He has given pre- 
sentations at JavaOne, Microsoft TechEd and TechReady, and numerous aca- 
demic and technical conferences, in addition to several MSDN Webcasts. 


FRANK COHEN Push To Test 


Frank Cohen is the leading authority for testing and optimiz- 

ing software developed with Web, SOA, AJAX, and REST 

designs and implementations. Frank Cohen is CEO and 

Founder of PushToTest and inventor of TestMaker, the open- 

source test automation tool. He is author of several books 
on optimizing information systems, and is an experienced 
instructor in areas relating to testing Web applications, SOA, REST, and AJAX. 
Frank was also a featured speaker at JavaOne, SOA World 2007, STARWest, PSQT, 
and has been a guest of SYS-CONTV. 


ROB DAIGNEAU SynXis 

Rob Daigneau has over eighteen years experience designing 
and implementing enterprise-class applications for a broad 
array of industries from manufacturing, to financial services, 
to retail and hospitality. Rob is currently the Chief Architect 

) for SynXis, the leader in distribution and messaging software 
technologies for the hospitality industry. He also has served 
as the Director of Application Architecture for Monster.com, one of the most vis- 
ited web sites in the world. Rob is currently working on a book of service design 
patterns for Addison Wesley. 


KATHLEEN DOLLARD GenDotNet 
Kathleen has been developing business applications for over 
20 years, programming in Visual Basic for almost ten years, 
and working with .NET since the early betas. As an independ- 
ent consultant, she has had the opportunity to work in a 
variety of domains, including the finance and justice sectors. 
Kathleen has worked extensively with application code gen- 
eration and is the author of Code Generation in Microsoft .NET (from Apress). 
She has published numerous articles on a range of .NET technologies including 
XSLT, debugging, ADO.NET, and code generation. Kathleen is also a long time 
Microsoft MVP, president of the Northern Colorado .NET SIG, and is an active 
member of the Denver Visual Studio User Group. 


DINO ESPOSITO IDesign, Inc. 

Dino works with IDesign, Inc., writes the “Cutting Edge” column 
| for MSDN Magazine, and regularly contributes to the MSDN 
ASP.NET DevCenter and a variety of developer magazines. Dino 
is also the author of the two volumes Programming 
Microsoft ASP.NET 2.0 (Microsoft Press, 2005). Get in touch 
at http:/Aweblogs.asp.net/despos. 


NEAL FORD ThoughtWorks 

Neal Ford is Software Architect and Meme Wrangler at 
ThoughtWorks, a global IT consultancy with an exclusive 
focus on end-to-end software development and delivery. He is 
also the designer and developer of applications, instructional 
materials, magazine articles, courseware, video/DVD presen- 
tations, and author and/or editor of 5 books spanning a vari- 
ety of technologies. He focuses on designing and building of large-scale enter- 
prise applications. He is also an internationally acclaimed speaker, speaking at 
over 100 developer conferences worldwide, delivering more than 600 talks. Get in 
touch at http://weblogs.asp.net/despos. 


RUSS GIBFRIED SAIC 


Russ Gibfried has almost 20 years of industry experience 
working on various enterprise-wide IT initiatives for clients 
such as Microsoft, Hewlett-Packard, Turner Broadcasting, 
Disney and various financial institutions. Mr. Gibfried's skills 
expand into many areas including Internet / extranet archi- 
tecture and development, secure content distribution, enter- 
prise integration and business workflow. As a Solutions Architect in SAIC’s 
Corporate IT department, Russ is responsible for evaluating and implementing 
technologies that define SAIC’s enterprise modernization activities. 
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CHRIS HADDAD Burton Group 


Chris Haddad is Vice President and Service Director of 
Burton Group's Application Platform Strategies Service. 
He leads research teams that focus on infrastructure 
technologies used to design, develop, and deploy applica- 
tions and services. His coverage areas include service- 
oriented architecture, model-driven development, business 
process management, open source, and technology adoption strategies. Chris 
has established software development processes, product direction, and inte- 
gration strategy of several successful software companies. He was granted 
committer status on the Apache Axis project in 2002, and served as an advi- 
sor on conference editorial boards and chair of special interest groups. Chris 
is an author, editor, and contributor on numerous articles, technical publica- 
tions, presentations, and podcasts. He is a frequent speaker at internal cor- 
porate events and industry conferences including Enterprise Architecture 
Summit, Networld+Interop, SOAWorld, JBossWorld, IASA, ITARC, and SDWest. 


ALLEN HOLUB Holub Associates 


Allen Holub has worked in the computer industry since 

1979 on everything from operating systems, to compilers, 

to web applications. He is a consultant, providing mentor- 

ing and training in 00-Design and Java development, 

technical due diligence, design reviews, and even writes 
programs on occasion. Allen was an early adopter of Java, 
programming in it since its release in 1995. He worked in C++ for eight years 
before that, and has also worked in C, Perl, Pascal, PL/M, FORTRAN, SQL, and 
various assembly languages. He learned design the hard way, by beating his 
head against programs that he'd rather not admit that he'd written, and is 
now a recognized expert in 00-Design, UML, and process. He served as a 
Chief Technology Officer at NetReliance, Inc—an internet-security-infrastruc- 
ture company—and sits on the boards of advisors for Ascenium Corp. and 
Ontometrics. He is also the Security-Track chair for the Software 
Development conference. Allen has authored nine books (including Holub on 
Patterns: Learning Design Patterns by Looking at Code, Taming Java 
Threads, and Compiler Design in C) and 100+ magazine articles (for SD 
Times, JavaWorld, Dr. Dobb's Journal, Programmers Journal, Byte, MSJ, 
and others). Allen has been a Contributing Editor for JavaWorld since 1998, 
and for SD Times since 2004. He wrote the popular “O0-Design Process” col- 
umn for the IBM developerWorks Component Zone, and was the technical edi- 
tor of CMP Media's Java Solutions. Allen has taught regularly for the 
University of California (Berkeley) Extension since 1983. Contact Allen at 
http://www.holub.com/allen.htm| 


RON JACOBS Microsoft 


Ron Jacobs is an Architect Evangelist in the Microsoft 
Architecture Strategy group based at the company head- 
quarters in Redmond Washington. Since 1999 Ron has 
been a product and program manager on various 
Microsoft products including the .NET Framework, 
Windows Communication Foundation and COM+. A top-rated 
conference speaker, author and host of the podcast show ARCast.TV, Ron 
brings over 20 years of industry experience to his role of helping Microsoft 
customers and partners to build architecturally sound and secure applications. 


ANANT KADYALA BEA/Oracle 


Anant Kadiyala is a SOA Solutions Architect at BEA 

Systems, where he advices BEA's key customers to define 

and implement their SOA Strategy. Anant is a frequent 

speaker at various forums and has published several arti- 

cles in various leading journals. Prior to BEA, Anant was an 
independent consultant helping companies implement SOA. 
He can be reached at anant.kadiyala@bea.com 


MICHELE LEROUX BUSTAMANTE IDesign, Inc. 
Michèle Leroux Bustamante is Chief Architect of IDesign Inc., 
Microsoft Regional Director for San Diego, and a Microsoft MVP 
for Connected Systems. At IDesign, Michèle provides training, 
mentoring and high-end architecture consulting services 
focusing on scalable and secure architecture design for .NET, 
web services, interoperability, federated security scenarios, 
CardSpace, and globalization architecture. Michèle participates in Software Design 
Reviews for products in the Microsoft roadmap, including WCF, CardSpace and 
other security-focused products. During the Beta 1 phase, Michèle participated in 
prototyping elements of the CardSpace technology for the product team. She is a 
member of the International NET Speakers Association (INETA), a frequent confer- 
ence presenter, conference chair for SD West, and is frequently published in sever- 
al major technology journals. Michèle is also on the board of directors for IASA 
(International Association of Software Architects), and a Program Advisor and 
instructor for UCSD Extension. Her latest book is Learning WCF (O'Reilly 2007). - 


see her book blog here: www.thatindigogirl.com. Reach her at mlb@idesign.net, or 
visit www.idesign.net and her main blog at www.dasblonde.net. 


ROCKFORD LHOTKA Magenic Technologies 

Rockford Lhotka is the author of several books, including 
Expert One-on-One Visual Basic .NET Business Objects 
by Apress, and is a columnist for MSDN Online and a con- 
tributing author for VSM. He speaks at major conferences 
around the world. Rockford is the Principal Technology 
Evangelist for Magenic Technologies, a Microsoft Gold 
Certified Partner. Reach Rocky at rocky@lhotka.net. 


JUVAL LOWY IDesign Inc. 

Juval Lowy is a software architect and the principal of 
[Design (www.idesign.net), specializing in NET 3.0 architec- 
ture consulting and advanced .NET 3.0 training. Juval is 
Microsoft's Regional Director for the Silicon Valley, working 
J with Microsoft on helping the industry adopt .NET 3.0. His lat- 
est book is Programming WCF Services (O'Reilly 2007). Juval 
participates in the Microsoft internal design reviews for future versions of NET 
and related technologies. Juval published numerous articles, regarding almost 
every aspect of NET development, and is a frequent presenter at development 
conferences. Microsoft recognized Juval as a Software Legend as one of the 
world's top .NET experts and industry leaders. 


ADNAN MASOOD Green Dot Corporation 

Adnan Masood works as a Sr. Software Engineer / Tech. 
Lead in a Monrovia based financial institution where he 
builds SOA-based enterprise applications using Microsoft 
.NET Framework. He is a regular speaker to technical con- 
ferences and code camps, is attributed and published in 
print media and on the web and holds a Masters Degree in 
Computer Science. Adnan is actively pursuing his doctoral degree in Machine 
Learning and is highly involved in the .NET community as co-founder of San 
Gabriel Valley NET Developers group. He holds various technical certifica- 
tions including MCPD.NET, MCSD.NET, and SCJP-II and is also a Microsoft 
Certified Trainer. His blog can be found at www.AdnanMasood.com and he can 
be reached via email at adnanmasood at gmail.com 


JAMES MCCAFFREY Volt Information Sciences, Inc. 
Dr. James McCaffrey works for Volt Information Sciences, 

| Inc., where he manages technical training for software engi- 
| neers working at Microsoft's Redmond, WA campus. He has 
worked on several Microsoft products including Internet 
Explorer and MSN Search. James is the author of .NET Test 
Automation Recipes (Apress, 2006) and is a contributing 
editor for Microsoft's MSDN Magazine. McCaffrey holds a doctorate from the 
University of Southern California, and degrees from the University of California, 
Irvine, and California State University, Fullerton. 
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TED NEWARD Neward and Associates 


Ted Neward is an independent software development 
architect and mentor in the Seattle, Washington area. He is 
the author of a number of books, including Effective 
Enterprise Java (Addison-Wesley), Server-Based Java 
Programming (Manning), and co-authored Shared Source 
CLI Essentials (O'Reilly) and C# In a Nutshell (O'Reilly) 

with Peter Drayton and Ben Albahari. He teaches and authors 
both Java and .NET, speaks frequently for technology user groups and writes 
technical papers available at.www.tedneward.com. 


N CURT PETERSON Neudesic 


Curt Peterson is a Distinguished Engineer at Neudesic, one 
of Microsoft's leading .NET professional services firms and 
j Microsoft Gold Certified Partners, headquartered in Irvine, 


CA. Curt brings considerable expertise and over twenty five 
years of experience in building complex, mission critical 
applications based on Microsoft technologies, and is respon- 

sible for all field engineering aspects of Neudesic’s Neuron ESB 
product. He has presented technical tracks at numerous industry events includ- 
ing Microsoft's Visual Studio.NET roll-outs and Gartner conferences. Curt is a 
member of the Southern California NET Architect Council and is a regular 
speaker and trainer for numerous Microsoft .NET programs, specializing in SOA 
and Enterprise Architecture. In addition, Curt was the founding member of the 
Microsoft Connected Systems Division Virtual Technical Specialist Team, and is 
a part of the Microsoft Connect Systems Advisory Board. 


DAVID PLATT Rolling Thunder Computing, Inc. 

David S. Platt teaches Programming .NET at Harvard 
University Extension School and at companies all over the 
world. He is the author of eleven programming books. His 
Introducing Microsoft .NET from Microsoft Press intro- 
duced thousands of programmers to that environment. Even 
today, 4 years after its most recent release, it is outselling 
Tom Clancy's Every Man a Tiger on Amazon.com, which tells you what kind of 
geeks buy their books there. His magnum opus, Why Software Sucks (Addison- 
Wesley, 2006, www.whysoftwaresucks.com), points out ways in which software 
MUST improve if it's to accompany humanity into the twenty-first century. He is 
famous for his engaging presentation style. “He's the only guy | know that can 
actually make a talk on COM's apartment threading model funny,” said one stu- 
dent. Microsoft named him a Software Legend in 2002. Dave holds the Master of 
Engineering degree from Dartmouth College. He did his undergraduate work at 
Colgate University. When he finishes working, he spends his free time working 
some more. He wonders whether he should tape down two of his daughter's fin- 
gers so she learns how to count in octal. He lives in Ipswich, MA. Nickname: “The 
Mad Professor” Favorite Web Site: cadomargaritaville.com Comment most fre- 
quently elicited from children at adjoining breakfast restaurant tables before 
he's had his morning coffee: “Mommy, what's wrong with that man?” 


PAUL PREISS IASA 
Paul received a bachelor's degree in Japanese from the 
University of Texas at Austin. He went on to become a project 
manager for Human Code. Later, Paul was the Applications 
Manager and Architect for Dell Pan Asia based in Kawasaki, 
Japan. He then became the Sr. Architect for a software con- 
sulting firm in St Paul Minnesota, where he provided architec- 
ture consulting for numerous government and private enterprise 
clients. Paul went on to become the Director of Engineering and Chief Architect of 
a digital asset management company. More recently Paul has spent most of his 
time creating and managing the International Association of Software Architects. 


DR. NEIL ROODYN 

Dr. Neil is an independent itinerant consultant, trainer and 
author. He travels the world working with software compa- 
nies. Dr. Neil loves Australia, where he spends the summer 
enjoying the Sydney lifestyle and helping software develop- 
ment teams get more productive. Dr. Neil spends his other 


summer each year flying between northern Europe and the USA working with 
software teams and writing about his experiences. Neil brings his business and 
technical skills to the companies he works with to ensure he has happy cus- 
tomers. His latest book eXtreme NET introduces eXtreme Programming tech- 
niques to .NET developers. Dr. Neil's latest venture is a Web site to help develop- 
ers build on Virtual Earth, you can find out more by visiting 

http://www ViaVirtualFarth.com. You can find out more about the man from his 
Web site http://www.Roodyn.com or you can email Dr. Neil at Neil@Roodyn.com 


RANDY SHOUP eBay 


Randy Shoup is a Distinguished Architect in the eBay 
Marketplace Architecture group. Since 2004, he has been 
the primary architect for eBay's search infrastructure. Prior 
to eBay, Randy was Chief Architect at Tumbleweed 
Communications, and has also held a variety of software 
development and architecture roles at Oracle and 
Informatica. He received a BS in Mathematical and Computational Science, and 
an AB in Political Science, from Stanford University. 


KEN SPENCER Solid Quality Mentors 

Ken is the Practice Manager, Development Platform and a 

mentor for Solid Quality Mentors. He has been developing 

and managing application development with various lan- 

guages and technologies for over two decades. He has also 

developed tools to migrate applications to .NET such as 
large Access applications. Ken spends his time these days 
providing software design guidance, working with clients on SharePoint and .NET 
technologies, working as build master for large projects (with our customer 
build software) and teaching in various venues, and works with a great team to 
provide custom technology and services. 
Ken has experience developing applications with Visual Studio NET and the NET 
Framework, WPF, Visual Basic, C#, and SQL Server. He specializes in enterprise 
collaboration and reporting solutions using SQL Server Reporting Services and 
SharePoint 2007. He was the team leader on producing the .NET Readiness train- 
ing for Microsoft to roll out the .NET Framework and has been part of the team 
Microsoft uses to deliver training on each version of .NET. Ken also sits on the 
Microsoft Software Development Roundtable that focuses on the design of 
future versions of .NET technology. Ken is also a Microsoft Regional Director. 


ANGELA YOECHEM Bank of America 


Angela Yochem runs the Strategic Architecture Management 

organization in one of the world’ s largest retail banks. Her 

teams are responsible for architectural direction, maturity, 
and technology as a strategic business enabler. 

Angela joined the bank from another large financial services 
organization, where she led multiple architecture practices 
and was responsible for architectural health, maintenance of disciplines and 
transformation based on the needs and objectives of the business and corporate 
technology. Prior to the banking industry, Angela held a senior role in the 
world's largest package delivery company, a leading global provider of special- 
ized transportation and logistics services. She specialized in the design and 
delivery of large-scale distributed systems and solutions to complex integration 
and convergence challenges. Her background includes extensive B2B and B2C 
commerce architectural work for the world’s largest information technology 
company, systems design for the defense industry, and network design and man- 
agement of a multi-campus network with over 5,000 managed nodes. 

Angela is one of nine Fellows worldwide associated with the International 
Association of Software Architects (IASA), recognized for her contributions to 
the Foundation Reference Model and Architecture Skills definitions. She is a pub- 
lished author with Prentice-Hall and Addison-Wesley, and serves on the executive 
board of Global Exec Women. She was nominated as Women in Technology's 
Woman of the Year in 2006. Angela is a regular speaker at events and 

executive forums in the United States and abroad. 
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the PRIMARY file group to another more-appropriate 
file group. 

For the stored procedure to work, the new file 
group must already exist and contain one or more files. 
Because sp_MoveClstTablesToFileGroup uses the sp_ 
MsForEachTable system stored procedure to populate 
a global temporary table with the sp_HelpIndex results 
on all user tables, the executing user must be at least 
a database owner (DBO) who is able to create global 
temporary tables and execute system stored procedures 
from the master database. 

Using sp_MoveClstTablesToFileGroup is easy. 
Suppose you want to change all the clustered-indexed 
user tables in the Pubs database from the PRIMARY 
file group to the NEWFG file group. Here are the steps 
you'd follow: 

1. Compile sp_MoveClstTablesToFileGroup inside the 
Pubs database. 

2. Log on to the server using a sysAdmin server role 
login or pubs dbOwner role. 


3. Execute the stored procedure using the code 


USE pubs 

GO 

EXEC sp_MoveClstTablesToFileGroup 
@newFi 1eGroup="NEWFG' 


4. After running the stored procedure, check to see 
whether all of the clustered-indexed user tables 
residing in the Pubs database moved to the NEWFG 
file group from the PRIMARY file group. 


I wrote sp_MoveClstTablesToFileGroup on SQL 
Server 2005 Developer Edition SP1, but it also works 
on SQL Server 2000. You can download the code file 
(ChangeObjectFG.sql) by going to www.sqlmag.com, 
entering 99204 in the InstantDoc ID text box, clicking 
Go, then clicking the 99204.zip hotlink. 

—Eli Leiba, senior application DBA, 
Israel Electric Company 
InstantDoc ID_99204 
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i ave you ever needed to quickly find all the 
stored procedures that contain a certain 
field? Have you ever needed to find some code 
you wrote a while ago but have forgotten the 
name of that code? Have you ever needed to 
find all the code on the server that references a 
particular table? To find such items, you could 
purchase database documentation software—or 
you could use Find.sql. 

Find.sql is the most popular piece of code 
I’ve ever written. This stored procedure finds 
any string in any type of SQL code (e.g., view, pro- 
cedure, function) in any database on a server without 
hurting the server’s overall performance. Find.sql 
won't search for strings in the database’s data, 
however. If you need that capability, check out the 


table-searching script at www.users.drew.edu/skass/ 


sql/SearchAlITables.sqL txt. 


I named the stored procedure Find so its name is 
easy to remember. Find.sql is also easy to use. You 
Just need to download the stored procedure by going 
to www.sqlmag.com, entering 99466 in the InstantDoc 
ID text box, clicking Go, then clicking the 99466.zip 
hotlink. On the server you want to search, you call the 
stored procedure by typing 


Find 'mystring' 


where mystring is the string you want to search for. 


DatabaseName ObjectName 

MYDB proc_my_summary 
MYDB 
YOURDB 
YOURDB 
PUBS 
PUBS 
PUBS 


Current_Search 
Searching For: ‘test’ 


ObjectType 
P 


mt_datarun 
prc_mytest 
e netitse 
fn_sales 
een kes 
ysales 


Figure 2 


Sample output from Find.sql 


The stored procedure will search for that string in 
all the databases on your server and return a list of 
the objects that contain it. As Figure 2 shows, the 
list is ordered by database. It tells you the name of 
the object as well as the type of object in which your 
string has been found. It uses notations such as P for 
procedure, FN for function, and V for view for the 
object type. 

I wrote Find.sql for SQL Server 2000, but it 
should work on SQL Server 6.5 through SQL Server 
2008. Many DBAs and developers are using it to 
make their jobs easier. Play around with Find.sql 
to see what it can do—it could save you hours of 
frustration when you need to find a specific piece of 
code. SQL 

—Michael Berry, senior DBA, 
Ohio Public Employees Retirement System 
InstantDoc ID_ 99466, 
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Sqlmag.com Community Dialog 


Videos and Reader Comments Give 
Sqimag.com Readers Online Help 


SQL Mag Experts... Now 
Appearing on ITTV.net! 

There’s a brand-new video Web site that has joined 
the Sqimag.com and Windowsitpro.com family: ITTV 
„net. On ITTV.net, you'll find video interviews with 
technical authorities and industry experts on a range 
of topics of interest to DBAs, developers, and IT pro- 
fessionals. For example, at TechEd 2008 in June, SOL 
Server Magazine Technical Director Michael Otey and 
contributing author Erin Welker had a video chat with 
Senior Editor Sheila Molnar about their impressions 
of the new business intelligence (BI) and data ware- 
housing features in SQL Server 2008. Here are some 
of the highlights of the conversation. 

“They’ve made some improvements on how parti- 
tions are processed during the queries,” Erin said. “And 
another thing I’m really excited about... is partition- 
aligned indexed views, which will make it a lot easier 
to implement aggregation, much like you would in 
Analysis Services.” 

“On the BI side, there have been some really 
interesting improvements for SQL Server Integration 
Services... some very nice performance enhancements 
there,’ Michael said. “The same is true with [SQL 
Server] Reporting Services. It’s quite a bit faster. And 
the integration of the Office components: I know they’ve 
made an agreement with SoftArtisans to bring part of 
their OfficeWriter product into Reporting Services, and 
I think those are all good moves to help surface out the 
information in SQL Server to the end users.” 

Check out the rest of the video at ittv.net/Video 
Player/tabid/57/VideoId/26/SQL-Server-Round-Table- 
Discussion-With-Mike-Otey--Erin-Welker.aspx. And 
keep an eye on the ITTV.net home page for the latest 
videos. 


Information Gems in Online 
Reader Comments 

You might like the reader comments section at the 
end of Sqlmag.com articles because it lets you give 
feedback to an article’s author, or you just might 
enjoy reading other people’s comments. But did you 
know that reader comments often contain helpful tips, 
answers to reader questions in comments, or links to 
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relevant Web resources? Here are a few examples of 
some of the pearls of wisdom that 
Sqlmag.com readers have passed along 
recently in their comments. 

From vidma, on Derek Comin- 
gore’s “SQL Server 2008: Enter the Data 
Collector,’ June 2008, InstantDoc ID 
99420: “I blogged about Management 
Data Warehouse and Data Collectors, 
and maybe my post will be also helpful 
to this article’s readers: www.ssas-info 
.com/VidasMatelisBlog/55_sql-server- 
2008-management-data-warehouse.” 

From Lee Everest, on Kalen Dela- 
ney’s “Knowledge and Wisdom: Honoring Jim Gray,” 
June 2008, InstantDoc ID 99379: “Nice article. I 
blogged about Dr. Gray in August 2007 and have a 
few other links and tidbits of interest. Great visionary 
for our profession, and will be missed. www.texastoo 
.com/post/Dr-Jim-Gray—Where-are-you.aspx.” 

From kgutlich, on Teo Lachev’s “Protect UDM 
with Dimension Data Security, Part 2,” October 2007, 
InstantDoc ID 96763: “I’ve tried to use the factless fact 
table as a security method.... When I insert my MDX 
expression (almost the same as the one in the example), 
the syntax is correct but I get an error using the OLAP 
cube telling me that the EXIST function does not exist 
while trying to use the security role.... Can anybody 
help me?” From carleb21, in response to kgutlich: “It 
looks like a simple typo. Try Exists instead of Exist.” 


MySQL vs. SQL Server 
Back in April, Brian Moran wrote about the signifi- 
cance of Sun Microsystems’ acquisition of MySQL and 
its potential effect on SQL Server. Readers chimed in 
with thoughtful comments about the strengths and 
weaknesses of MySQL and SQL Server, and Brian 
responded to update and clarify some of the points in 
the original article. In this case, the reader comments are 
almost an article in themselves! Read them for yourself 
at InstantDoc ID 98951. [SQL 
—Anne Grubb, Web site strategic editor, 
SQL Server Magazine 
InstantDoc ID 99534 


GET ACTIVE ONLINE AT 
SQLMAG.COM! 

What features do you want to see 
in SQL Server 2008? A SQL Server 
edition that runs on UNIX? A trans- 
action-log reader? Integrated forms? 
Technical Director Michael Otey 
asks that question in his editorial 
“What Readers Want SQL Server 
to Include,” InstantDoc ID 98866. 
Tell him what you think! 
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MSSQL Blocks and Orphan Finder 


These free tools help you find problems 
with your SQL Server instances and database tables 


Kevin Kline 
kevin.kline @ quest.com) is the director of 


technology for SQL Server Solutions at Quest 
Software and a founding board member of 
the international Professional Association for 
SQL Server. He is the author of SQL in a 
Nutshell, 2nd edition (O'Reilly Media, 2004). 


Figure | 


The Wait Info column 
in the MSSQL Blocks 
interface 


14 August 2008 


ypically, I discuss only one free tool each month in 

my Tool Time column. However, I have two free 
single-purpose utilities to discuss this month. MSSQL 
Blocks, which was written by Konstantin Knyazev, a 
SQL Server and Delphi enthusiast living in Moscow, 
makes it easy to spot locking and blocking problems 
across multiple SQL Server instances. Orphan Finder, 
which was written by Miljen Stankovic, a multicertified 
SQL Server DBA and development manager working 
in Zagreb, Croatia, finds foreign key columns that don’t 
have a matching value in the parent key. Let’s take a 
look at what each of these tools does in more detail. 


MSSQL Blocks 

It can be difficult to use SQL Server's native tools 
to determine which server in a server farm might be 
having blocking problems because SQL Server’s native 


(0.48) vec - as Di f 

(0.453 sec - PAGEICLATCH_SH (5 1.444840} 

(0.453 one - PMGEIOLATCH EH (5 1.444844} 
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tools work on only one server at a time. However, 
MSSQL Blocks makes it quick and easy to determine 
which SQL Server instances are having blocking prob- 
lems. MSSQL Blocks is a Delphi-based utility that 
collects information (via ADO in asynchronous mode) 
about blocked processes from multiple SQL Server 
2005 and 2000 instances, and presents the information 
in an intuitive format. This utility is especially useful 
if your environment includes applications that have 
transactions that span multiple SQL Server instances. 
The format of information retrieved in MSSQL Blocks 
is based on the server ID and the server process ID 
(SPID). The tool also reports the wait state, which 
describes the type of lock held by the SPID, in the 
Wait Info column of the MSSQL Blocks console, as 
shown in Figure 1. 

In addition to reporting the current locking and 


blocking activity on all the monitored servers, MSSQL 
Blocks also stores the history of blocking activity on 
the SQL Server instances in an XML file. This history 
can be very helpful if you want to analyze blocking 
trends and behaviors on one or more servers over a 
long period of time. 


Orphan Finder 

This simple GUI tool helps you identify records in a 
SQL Server 2005 database that have values in a foreign 
key column that don’t exist in the parent table. Orphan 
Finder can be especially useful when analyzing data 
that originated in legacy systems or in systems that lack 
rigorous data-cleansing capabilities. 

To use Orphan Finder, enter the connection details 
of the SQL Server instance and database that you 
want to analyze. Note that you must know the name 
of the database before you attempt to connect to it 
because there’s no drop-down list of database names 
to choose from. Once you connect to the SQL Server 
database, Orphan Finder analyzes the database and 
displays all the database tables and their foreign keys in 
a tree structure. Errors in data quality are highlighted 
in red in the report, as shown in Figure 2. To see all 
the database records that don’t have a parent record, 
double-click the name of the foreign key containing 
the error and Orphan Finder will expand the details to 
show all the records containing errors. 


MSSQL BLOCKS 
and ORPHAN FINDER 


BENEFITS: MSSQL Blocks helps you determine 
which SQL Server instances in your organization 
are having locking and blocking problems. Orphan 
Finder helps you find records in your SQL Server 
2005 databases that have values in a foreign key 
column that don’t exist in the parent table. 


SYSTEM REQUIREMENTS AND NOTES: MSSQL 
Blocks requires SQL Server 2005, 2000. Orphan 
Finder requires SQL Server 2005 and the Microsoft 
.NET Framework. 


HOW TO GET IT: You can download MSSQL Blocks 
from sqlblocks.narod.ru. You can download Orphan 


Finder from www.spi.hr/ZalTprofesionalce/Orphan 
Finder/tabid/356/Default.aspx. 
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MSSQL BLOCKS AND ORPHAN FINDER ote 


System 
Requirements 
MSSQL Blocks is a 
standalone product that 
can connect to SQL 
Server 2005 and 2000 and 
requires the client connec- 
tion tools that ship with 
SQL Server Management 
Studio (SSMS). You 
can download MSSQL 
Blocks from sqlblocks 
.narod.ru. Konstantin also 
supports the tool via the 
MSSQL Blocks Google 
group at groups.google 
.com/group/sql-blocks. 
Orphan Finder is a 
Microsoft .NET application that currently connects 
to only SQL Server 2005 databases. It’s available 
for download from www.spi.hr/ZalTprofesionalce/ 
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Finder’s report 


We want to hear your feedback on the Tool Time 


OrphanFinder/tabid/356/Default.aspx, where Miljen discussion forum at sqlforums.windowsitpro.com/web/ 
DEEE A TE forum/categories.aspx?catid=169&entercat=y. 
supports the tool directly through email. SQL 
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Audit Data changes, 
Recover without a backup 


the Ultimate Log Reading, Auditing and 
Recovery tool for SQL Server 


Y Recover truncated, deleted or modified data 
Y Recover data from corrupted database files 
~ Recover Dropped tables 


Y Analyze row history for historical changes 
Y Generate UNDO and REDO scripts 
Y Export transactions to XML, BULK SQL etc 


yf Selectively recover data without a backup. fe ApexSat 


software 


SQL Server Magazine e www.sqlmag.com 


Wwww.apexsql.com 
or phone 866-665-5500 


August 2008 


Errors in data quality high- 
lighted in red in Orphan 


15 


These products will help you 


Squeeze the most out of SOL Server 


by the 
SQL Server Magazine Editors 


16 August 2008 
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BEST 


PRODUCT OF THE YEAR 


Intel ° www.intel.com 


Few products have the ability to turn an industry on 
its ear, but that’s exactly what Intel has done with the 
Quad-Core Xeon Processor 5400 Series. Massive data- 
base servers and the sprawling data-driven applications that depend on them can 
bring puny single- and dual-core processors to their wobbly, silicon knees. AMD, 
Intel’s primary rival, put forth a competing solution that suffered from produc- 
tion delays and setbacks, leaving Intel as the clear market leader in this burgeon- 
ing high-end segment of the CPU market. The benchmark-crushing performance 
that Intel’s Quad-Core CPUs provide can result in huge cost savings for DBAs 
tasked with squeezing maximum performance out of enormous (and enormously 
complex) databases. 

Shannon Poulin, director of the Server Platforms Marketing Group at Intel, says 
that his company worked closely with the SQL Server software community to ensure 
that Intel’s architecture worked with parallelized applications such as SQL Server 
2005 and that reduced power consumption and increased performance will continue 
hand-in-hand. “We've delivered an increasing amount of performance in the same 
energy consumption envelope,” says Poulin. 

SQL Server Magazine Technical Director Michael Otey agrees, pointing out that 
Intel’s Quad-Core processors allow certain applications to take advantage of that 
extra processing power. “Enterprise-level database servers such as Microsoft SQL 
Server 2005 and SQL Server 2008 can take full advantage of all CPUs that are pres- 
ent, as can virtualization software,” says Otey. “Because these types of applications 
are designed for multiprocessor support, they can initiate separate threads on the 
individual processors of multicore systems.” 

The rapid growth of virtualization technology and 
the imminent arrival of SQL Server 2008 might be 
garnering lots of attention these days, but you can 
count on this: Most of those impressive new soft- 
ware applications will be powered by more than a 
few burly Intel Quad-Core processors. 
—Jeff James 


Shannon Poulin, director of the Intel 
Server Platforms Marketing Group 
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Microsoft 
www.microsoft.com 


EDITORS’ 


BEST 


BREAKTHROUGH PRODUCT 


Although lots of media at- 
tention has been focused 
on the release of Windows 
Server 2008 and SQL Server 
2008, Microsoft Visual Stu- 
dio (VS) 2008 had the distinction of hitting the mar- 
ket first. The latest version of Microsoft’s ubiquitous 
IDE offers dozens of improvements and features for 
programmers, including .NET Framework improve- 
ments, support for updated versions of C# and Visual 
Basic, and an updated version of Microsoft Founda- 
tion Classes (MFC 9.0). 

For the SQL Server community, the most signifi- 
cant new feature in VS 2008 is its support for Lan- 
guage Integrated Query (LINQ), a Microsoft .NET 
Framework component that introduces data query- 
ing to .NET languages. It’s big news for database 


SPECIAL AG aaa 


Microsoft 
www.microsoft.com 


With Microsoft System Cen- 
ter Data Protection Manager 
(DPM) 2007, SQL Server 
data protection takes a great 
leap forward. Because DPM 
2007 is integrated with SQL 
Server, DBAs can use a native Microsoft product to 
perform database backup and restore and protect a 
range of server systems under a common framework. 
Administrators no longer need to turn to one solu- 
tion for long-term archiving, another for rapid recov- 
ery, and a third for offsite backup and restore. 

Ted Kummert, corporate vice president of Micro- 
soft’s Data Storage and Protection Platform Division, 
notes that Microsoft “added enterprise-class capabili- 


EDITORS’ 


BEST 


SPECIAL ACHIEVEMENT 


SQL Server Magazine * www.sqlmag.com 


developers, and many think it could change the way 
SQL Server databases are developed. 

According to Gert Drapers, group engineering 
manager on the VS 2008 team, feed- 
back from SQL Server developers has 
been positive. “LINQ and the entity 
data model are popular,” says Drapers. 
“There’s now less distance between the 
application and data tiers, which helps in 
overall application development.” 

“LINQ is quite possibly the biggest 
paradigm change for database developers 
since the advent of ODBC,” says Michael 
Otey. “LINQ eliminates the language- 
database disconnect and enables object- 
oriented database access through SQL-like extensions 
to VB or C#. Considering that LINQ is still in its early 
stages, it might be a while before you have to start wor- 
rying about converting your applications to use LINQ, 
but there’s little doubt that LINQ and DLinq represent 
the future of database development.” 

Jeff James 


ties such as recovery from disk and archive 
to tape. With DPM 2007 we also went one 
step further to design the protection from 
the inside out, so it would appeal to admin- 
istrators of specific workloads.” 

In “How DPM 2007 Fits into Your 
Data Protection Landscape” (www.sqlmag 
.com, InstantDoc ID 97347), Brian Moran 
notes that “Historically, most enterprise 
and midmarket customers have had to rely on sev- 
eral third-party offerings to manage their SQL Server 
backup infrastructure. [B]ackup hasn’t really been 
seen as the cool and sexy part of database 
management.... Don’t tell my wife, 
but DPM 2007 seems pretty 
hot to me.” 
—Sheila Molnar 


Gert Drapers, group 
engineering manager on 
VS 2008 


Ted Kummert, 
corporate vice president 
of the Microsoft Data 
Storage and Protection 
Platform Division 
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GOLD 


18 August 2008 


SECURITVZAUDITING 
and GOMPLIANG WINNERS 


SecureSphere Database Security Gateway 
Imperva ° www.imperva.com 


Why it won: SecureSphere Database Security Gate- 
way gives you complete visibility and control over 
your database applications by providing automated 
database activity monitoring, auditing, and compli- 
ance capabilities. 


Over the past few years, attacks on information systems 
have shifted from broad attacks on entire IT systems to 
more targeted attacks on business-critical data. Conse- 
quently, you need to know at all times who is access- 
ing your organization’s data and how it’s being used. 
This year’s security, auditing, and compliance Editors’ 
Best award winner, Imperva’s SecureSphere Database 
Security Gateway, addresses database security threats 
by providing visibility and control over all business- 
critical data in your organization. 

According to Mark Kraynak, senior director of 


o PEMA 


SECURE 


PHERE 


marketing at Imperva, “Database Security Gateway 
includes the visibility for database activity and appli- 
cation activity and can match those things up. So we 
can say, for a given transaction, this is the user, this is 
what they did in the application (asked for a change 
to data), and this is the change that got made.” 
SecureSphere Database Security Gateway moni- 
tors database activity and uses Imperva’s Dynamic 
Profiling technology to automatically create user pro- 
files and security policies according to user behavior. 
Integrated auditing and reporting capabilities also 
make it easy to prove compliance with government 
regulations. “It’s got a lot of nice automated features, 
it was very easy to get up and running, and it’s very 
simple to maintain,” said Kevin Alwood, a systems 
manager at Intuition Systems. 
—Megan Bearly 


SQLsecure 
solace Idera ° www.idera.com 

Why it won: SQLsecure provides a com- 
plete view of the security of your SQL 
Server instances and lets you easily analyze user and 
group permissions and create custom policies to 
monitor compliance with security standards. 


ApexSQL Log 
ApexSQL ° www.apexsql.com 


Why it won: In addition to letting you eas- 
ily audit SQL Server database and transac- 
tion log files for changes, ApexSQL Log can recover 
lost and modified data. 
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GET TO WHERE YOU WANT TO BE - FASTER. 


As your business intelligence (BI) needs change, fast track 
your development efforts by adopting a single, scalable 
platform based on the trusted technology of Crystal Reports®. 
Reduce your coding requirements with a complete report 
creation, management, distribution and viewing solution 


and experience the ease and confidence that comes with 


choosing one vendor for all your BI deployment needs. 


Choose the flexible BI tools and deployment options you need to get to where you want to be. 
Visit www.businessobjects.com/oneplatform today for your free trial or call us at 1-888-333-6007. 


Copyright © 2008 Business Objects SA. All rights reserved. Business Objects and the Business Objects logo and Crystal Reports are trademarks or registered trademarks of 
Business Objects SA or its affiliated companies in the United States and/or other countries. SAP is a registered trademark of SAP AG in Germany and in other countries. 
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BACKUP and AGOAN WINNERS 


Gold: 

SQL Backup 

Red Gate Software 
www.redgate.com 


Eqe[¥Server 
EDITORS’ 


Why it won: SQL Backup delivers 
market-leading compression, encryp- 
tion, and monitoring of SQL Server 
backups, making them fast, simple, and secure. 


SQL Backup 5 continues to give DBAs a robust 
backup tool, with wizards that make the GUI easy 
to use. Administrators can use Timeline Monitoring 
from the GUI to visualize and change the proper- 
ties of activities. According to Colin Millerchip, Red 
Gate Software’s head of product management, “SQL 
Backup 5 is a result of Red Gate’s attention to our 
customers’ needs. Discussions with DBAs revealed 
the need for a view of backup activity across the en- 
tire enterprise; this resulted in the Timeline Monitor- 
ing capability which was part of our VS release.” 

One feature that John Daniels, DBA for RBC 
Ministries, likes about version 5 is “the ability to up- 
grade all of my servers from my workstation. The 
process has been flawless.” 

Alan Greenstein, CIO of Hoss Equipment Com- 
pany, appreciates the fact that his team can “take 
snapshots throughout the day in about half the time 


of a Microsoft native backup, while re- 
ducing the impact on our users.” He 
notes that “our development staff have 
benefited by being able to quickly restore 
a database from production into a test 
environment. It is now possible for us to 
transmit one of our larger database back- 
ups to an offsite facility every night.” 
—Sheila Molnar 


redgete 


SQL Backup™ 


Compression and enorypten for BOL hack pes 


Silver: 
SQLsafe 
Idera ° www.idera.com 


HiServer 


of Idera’s SQLsafe provides a classic UI 
and high-speed backups that can be as much as 50 
percent faster than native backups. 


Bronze: 
LiteSpeed for SQL Server 


coe Quest Software e www.quest.com 


EDITORS’ ~~ 

JAYE Why it won: Quest Software’s LiteSpéed = 
wal for SQL Server helps companies reduee . 

reliance on tape and provides fast backups 

with a small, low-impact footprint. 
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Crystal Reports 2008 
Business Objects ° http://www.businessobjects.com 


Eqr[EServer 


Why it won: The industry leader has come through 
again with a comprehensive package that extends 
the ease of use of its enterprise-class, cross-plat- 
form reporting solution to end users. 


EDITORS’ 


BEST 


Crystal Reports 2008 moves the dial forward for enter- 
prise reporting software with a significantly improved 
end-user experience that includes new visualizations, 
what-if scenario modeling, and on-report interactive 
parameters. The 2008 version also makes it easier for 
developers to create applications that meld reports to 
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EDITORS’ BEST 


business practices and helps designers create reports 
that can be reused for various audiences. 

“T believe that Crystal Reports 2008 will change 
the way organizations use reporting,” said Peter Cal- 
laghan, chief sales officer at Maximizer. “Now that 
Flash can be embedded within a report, the report 
itself is transformed into a more sophisticated collab- 
oration tool, and this new interactivity has thousands 
of innovative potential uses. With Flash integration, 
the only limit for our use of Crystal Reports 2008 is 
our imagination.” 

Rob Hershfield, president of United Data Strat- 
egies, agrees. “Crystal Reports 2008 has leapfrogged 
everything else on the market. My future as a report- 
ing consultant has never been better, because now I 
can do so much more with the product.” 

—Sheila Molnar 


Silver: 

Tableau 

Tableau Software ° www.tableausoftware. 
com 
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Why it won: Tableau 3.0’s ease of use lets 
users of varying technical abilities analyze and moni- 
tor data simply by dragging and dropping fields to 
create dashboards. 


Bronze: 
Stimulsoft Reports.Net 2008.1 
Stimulsoft ° www.stimulsoft.com 


ElServer 
EDITORS’ 


Why it won: Stimulsoft Reports.Net 2008 
is a high-end tool for developers who need 
to generate reports in Microsoft .NET. The 2008.1 
version includes new Gantt and scatter charts, en- 
hances report exporting, and adds functionality to 
the report engine. 


WINNERS 


Gold: 


SQL mobile manager 
Idera ° www.idera.com 


Why it won: SQL mobile manager’s use of mobile 
management technology offers the potential for 
nimble troubleshooting and improved response time, 
helping DBAs reduce database downtime and ensure 
that small events don’t escalate into large ones. 


The simple phrase “SQL Server database manage- 
ment” actually reflects all the hours of a DBA’s life 
that are spent doing change and configuration man- 
agement, auditing, performance monitoring, backup 
and recovery, problem resolution, administration, 
and capacity planning. The big management suites 
help you manage everything from a central location, 
but Idera’s SQL mobile manager helps you manage 
no matter where you are. By harnessing mobile tech- 
nology to offer DBAs flexibility and convenience, 
SQL mobile manage claims our Editors’ Best award 
in database management. 

As the use of mobile devices to manage IT systems 
evolves, the resulting efficiency improvements could 
actually result in less time spent managing SQL Server. 
Certainly, mobile management helps you react quickly 
when performance plummets or other problems arise. 

Development manager David Bentz confirmed the 


product's flexibility. “With the instant alerts, monitor- 
ing, diagnostic details, and management features that 
SQL mobile manager provides, we can diagnose and 
correct SQL Server problems before users even no- 
tice. We really like being able to monitor our servers 
from anywhere 24 x 7.” 

—Caroline Marwitz 


Silver: 
Change Director for SQL Server 
Quest Software ° www.quest.com 


ElServer 
EDITORS’ 


Why it won: Change Director for SQL 
Server’s Change Tracker and Log Reader 
components are particularly well conceived. Change 
Tracker offers real-time alerts about database chang- 
es, and Log Reader provides an easy-to-use way to 
roll back unwanted changes. 
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Bronze: 

SQL Sentry Event Manager 

SQL Sentry ° www.sqlsentry.net 

EDITORS’ Why it won: SQL Sentry Event Manager 

BEST lets DBAs see, understand, and manage 
E SQL Server efficiently. Its unique cross- 

system job scheduling and monitoring capabilities 

make it a stand-out in the market. 
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Alternative Thinking About Microsoft Business Intelligence: 


BEFORE YOU CHOOSE 
HP FOR MICROSOFT BI, 


BE SURE TO CHECK 
OUR REFERENCES. 


Customers implementing business intelligence solutions 
using Microsoft® SQL Server™ need choice, performance, 
and scalable systems. Only HP Proliant solutions backed 
by HP BI expertise give you the trusted, scalable, and 
intelligent data platform you need to run your most 
demanding mission critical applications, reduce the time 
and cost of development & management of applications, 
and deliver actionable insight to your entire organization. 
Small wonder Microsoft named HP as a reference platform 
for Microsoft BI technologies. We're confident you'll want 
to give us a glowing referral, too. 


Technology for better business outcomes. 


Powered by the Quad-Core Intel® Xeon® Processor! 


1-866-356-6088 


Get down to the smallest details. Visit www.hp.com/solutions/microsott/sqlbi ip 


1, Intel, the Intel logo, Xeon, and Xeon Inside are trademarks or registered trademarks of Intel Corporation in the U.S. and other countries. 
The information contained herein is subject to change without notice. © 2008 Hewlett-Packard Development Company, L.P. 
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Gold: 
SQL diagnostic manager 
Idera ° www.idera.com 


Why it won: SQL diagnostic manager lets you eas- 
ily monitor and manage the health, performance, and 
availability of your SQL Server instances. 


When a SQL Server instance goes down or performs 
poorly, it can have a devastating impact on your IT 
infrastructure. You need to be able to easily monitor 
a server’s performance, health, and availability at all 
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times, as well as promptly diagnose and resolve per- 
formance problems. Idera’s SQL diagnostic manager 
(SQLdm) provides all the tools necessary to proac- 
tively monitor your entire SQL Server environment 
in real time so you can address database performance 
problems as quickly as possible. 

In his review “SQL diagnostic manager 5.0,” Feb- 
ruary 2008, InstantDoc ID 97831, John Green wrote, 


“SQLdm 5.0 includes everything you need to detect 
and diagnose most performance problems affecting 
your SQL Server instances.” The product’s easy-to- 
use UI lets you monitor multiple SQL Server instanc- 
es from one console and analyze performance trends 
through predefined reports. SQLdm is agentless and 
includes customizable alerts and notifications, as well 
as a centralized data repository. 

“Tt’s very easy to use. It gives you all the infor- 
mation you want without having to load agents 
everywhere,” says SQL Server DBA Patrick Shee- 
han. “It’s a very small footprint on my SQL Server 
systems.” 

—Megan Bearly 


Silver: 

Spotlight on SQL Server Enterprise 
sole Quest Software ° www.quest.com 

EDITORS’ 

Why it won: Spotlight on SQL Server En- 
terprise not only offers real-time perfor- 
mance metrics that let you easily find and diagnose 
performance problems—it also lets you automate 
problem resolution. 


Bronze: 
Embarcadero Performance Center 
Embarcadero ° www.embarcadero.com 


ElServer 
EDITORS’ 


Why it won: With Embarcadero Performance 
Center, you can monitor your databases 
from any computer that has Internet access, and its cus- 
tomizable alert thresholds and historical reporting help 
keep your SQL Server instances running at maximum 
performance. 


DEVELOPMENT PLATFORM WINNERS 
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Gold: 
XMLSpy 2008 
Altova ° www.altova.com 


Why it won: XML has become a core infrastruc- 
ture technology for many companies and products. 
Altova XMLSpy is the leading XML development 
platform and earns the gold in our development 
platform category. 


Some SQL Server developers work with XML fre- 
quently, particularly if they must integrate SQL 
Server with Web services. Altova XMLSpy 2008 has 
emerged as one of the preeminent XML develop- 
ment platforms and offers the editing and develop- 
ment features XML-savvy SQL Server developers 
need to be able to power through their tasks. “XML 
has become a core infrastructure technology for 
many companies and products, and Altova XML- 
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CONSIDERATIONS 
for Sizing a 
Microsoft SQL Server 


BI Solution 


Building an effective Business Intelligence (BI) infrastructure can be a tricky and 
complex process. Getting the right infrastructure in place requires that you have 
a detailed understanding not only of your own business processes, but also of 
what factors are important in building the underlying hardware infrastructure 
for a fast and effective BI solution. Here are the top 8 factors you should 
consider when choosing a BI solution. 


Understand the difference between other “sizing tools” 

and the HP SQL Server BI sizing tool 

Major differences exist among sizing tools available from a variety of hardware vendors. In most 
cases, the tools aren't actual sizing tools; rather they are reference configuration selectors. These 
tools ask a limited series of questions then point the user to a selection from a pre-existing list 
of reference configurations. However, a tool like the HP SQL Server BI sizing tool provides a much 
more realistic and effective approach because it actually “models” the workload for both servers 
and storage defined to the tool. The tool's algorithms were derived from real-world, benchmark- 
measured workloads and the tool uses that information to provide a reasonable workload 
configuration to meet a customer's requirements entered into the tool. 

To download the HP-BI-Sizer.for Microsoft SQL Server 2005, go to 


http:/www.hp.com/solutions/activeanswers/microsoft/sq! 
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Understand the difference between sizing 

for the three SQL Server BI tiers 

The three tiers of a SQL Server BI solution—ETL (Extract, Transformation, and Load), ROLAP 
(Relational Online Analytical Processing) and queries against OLAP (online Analytical 
Processing)—each have their own workload characteristics. Therefore, the HP SQL Server BI 
sizing tool models each workload differently and proposes a configuration to meet each tier's 
requirements. 


Sizing the ETL tier—Basic requirements 

To properly size the ETL tier, you must have a good feel for the number of rows to be loaded per 
day and the time window in which the load must be accomplished. If you plan to do weekly, 
rather than daily, loads you should expect that more rows will be loaded. This means that weekly 
batch windows will be more resource intensive or the ETL batch jobs will run longer because it is 
processing 7 days of data in a single batch window. You also need to understand the average size 
of each fact table and the expected number of partitions. Index creation and maintenance is also 
an important consideration when estimating hardware resources required to meet your service 
level agreements. Therefore, you should plan on partitioning the fact table and large dimension 
tables. Bear in mind that in many cases, maintaining indexes during fact table inserts may be 
more efficient than rebuilding the indexes against the fact table after a bulk data load. 


ADVERTISING SUPPLEMENT sponsored by HEWLETT PACKARD 


hp 


invent 


Adequate disk storage is critical 

Make sure that the ETL tier has sufficient disk storage to meet staging area requirements in both 
space and 1/0 throughput. Even though the staging area, database, tempdb, and transaction 

log may physically fit on only a few disk drives, HP takes special care in configuring your-1/0 
subsystem to support the expected throughput that the processors can generate. In other words, 
the HP sizing tool and reference configurations provide a balanced “processor to storage ratio” 
to provide users with excellent performance based upon their database size and workload. While 
internal storage may be used for such things as transaction logs,.an external storage device will 
make it easier to grow your solution. 


Key points for sizing the ROLAP tier 

Hardware requirements for the ROLAP tier are primarily driven by the number of queries per hour 
that your user population will generate and by the mix of query complexity. Large numbers of 
spindles provide higher performance; when using lots of disks, storage types like HP’s Enterprise 
Virtual Arrays provide for maximum flexibility, availability, and ease of management. In addition, 
as the number of disks grows to support higher volumes, your availability requirements will tend 
to increase. These features are HP’s EVA and XP SAN’s strong points. 


Key points for sizing the OLAP tier 

An understanding of query volume (queries per hour), complexity, and workload mix will be 
useful in determining the hardware requirements for the OLAP tier. For optimal performance, you 
should configure enough memory on the server to ensure that the most frequently used cells are 
memory resident. Because this user profile is sometimes difficult to determine during OLAP cube 
development, you may want to consider maintaining 100% of the OLAP cube as memory resident 
to ensure optimal performance. 


Understand that BI hardware requirements 

are very different from OLTP hardware requirements 

Administrators familiar with traditional OLTP database systems will understand that OLTP 
applications are IOPS sensitive because of the small size of each I/O (<=8KB). They want to support 
small I/O transfers at high I/O rates. Conversely, BI applications have a different workload profile. 
They tend to have fewer I/Os per second, but each I/O may be performing large block reads (300KB 
— 400KB-+ in size). Therefore, the system needs to support high gigabyte per second rates because 
throughput tends to be more important to BI applications than the number of I/Os per second. 


Understand your hardware expenses 

A large portion of a BI solution's initial costs will be for storage technology. More spindles 

means better performance and most BI installations will be designed around storage needs. The 
next investment is the server platform. This important decision will be determined by workload 
requirements as well as overall cost of ownership, energy requirements, and management 
capabilities. Customers look to HP’s broad server portfolio for the greatest choice in x86 and 
64-bit systems. From 1U to 8-socket designs, the ProLiant family of servers meets the needs 

for everything from business-critical, entry-level workloads to high-performance computing 
environments that demand reliable, scalable, and affordable server technology. As customer 
business demands grow and the need for more robust, mission-critical BI solutions expand, the HP 
Integrity product family delivers trusted performance that allows customers to scale up within the 
same footprint to keep server sprawl under control. 


HP understands the need to turn your data into information that drives business 
success. Contact HP for help in establishing a scalable and highly available 
infrastructure that supports your BI needs. 


hp 
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Spy is the leading XML development platform,” 
says Michael Otey. “Many [SQL Server develop- 
ers] use XMLSpy for developing XML documents 
and style sheets, as well as creating and debugging 
Web services.” 

XMLSpy 2008 includes all the basic editing, 
debugging, and profiling features that develop- 
ers need in an XML editor, as well as tight inte- 
gration with Visual Studio 2008. When database 
integration is required, XMLSpy doesn’t disap- 
point. It offers support for the most popular re- 
lational databases in their native languages, easily 
imports and exports data, and can use existing 
database structures to quickly create XML sche- 
mas. All these features combine to make XML- 
Spy 2008 a must-buy for SQL Server developers 
using XML. 

—Jeff James 


Silver: 

InstallShield 2008 Premier 
Elise Acresso Software ° www.acresso.com 
EDITORS’ 

Why it won: All but the simplest applica- 


tions need an installation routine, and 
most software vendors choose InstallShield as their 
tool. The new release supports the latest Microsoft 
technologies, including .NET 3.0, Windows Installer 
3.0, SQL Server 2005, and Windows Vista. 
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Bronze: 
JBuilder 2008 Enterprise 
CodeGear ° www.codegear.com 


EfellServer 
EDITORS’ 


BEST 


Why it won: Recently acquired by Embar- 
cadero, JBuilder 2008 Enterprise offers all 
the capabilities you’ve come to expect from Microsoft 
Visual Studio—except it’s for Java. If you need a Java 
development platform, JBuilder is hard to beat. 


DAV ELO PM TOOLS WINNERS 


Gold: 
NetAdvantage for .NET 
Infragistics 
www.infragistics.com 


ElServer 
EDITORS 


BEST 


Why it won: NetAdvantage is more 
than just a collection of controls; it 
provides a framework in which all 
the included components share similar object models 
and a common architecture. 


Building a .NET application can be time-consuming, 
particularly if you're starting from scratch. Predefined 
tools and controls can streamline the development 
process, but dealing with dozens of different tools 
from a plethora of disparate vendors adds another 
order of complexity to your development efforts. 
That’s where Infragistics’ NetAdvantage for .NET 
comes in handy. Although it offers several controls 
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for developers to use, it also provides a 
consistent, intuitive way to use them. “Net- 
Advantage goes beyond a collection of con- 
trols,” says Michael Otey. “It also provides a 
framework where all the components share 
similar objects and models and a common 
architecture. All these features combine to 
make these tools easy to learn and use.” 
Other program features that contribute 
to the usefulness of that framework include 
an export function (for PDF, .xls, and XML 
Paper Specification—XPS—formats), a cli- 
ent-side object model, and Microsoft Share- 
Point Portal Server support. Infragistics 
has also developed the Akido Framework, 
a collection of lightweight Asynchronous 
JavaScript and XML controls that integrate 
with the existing NetAdvantage control set. 
—Jeff James 
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EDITORS’ BEST 


Silver: 
ComponentOne Studio Enterprise 
EfrliServer ComponentSource 


Deg WWW-componentone.com 


am Why it won: For those who need to de- 
velop .NET apps in a hurry, ComponentOne Studio 
Enterprise is a good place to start. If you can’t find 
the component you need in this product bundle, you 
probably didn’t need it in the first place. 
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Gold: 
HP ProLiant DL580 G5 


HP ° www.hp.com 


Why it won: HP’s ProLiant servers have always been 
performance workhorses, but the HP Proliant DL580 
G5’s Quad-Core Xeon processor is an evolutionary 
step forward, with particular enticements for SQL 
Server admins. 


Intel’s Quad-Core processor family has shaken up SQL 
Server’s performance potential, as evidenced by the 
chip’s Product of the Year designation (page 16). The 
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most impressive implementation of the chip that we’ve 
seen is HP’s heavyweight ProLiant DL580 G5 server, 
an energy-efficient powerhouse that’s ideal for running 
data-intensive business applications, such as databases 
and business intelligence (BI) applications. In OLTP 
testing, the DL580 G5 delivers the world’s fastest indus- 
try-standard server performance, with a record-break- 
ing TPC-C benchmark. The DL580 G5 also boasts 
double the memory capacity of its predecessor. 

Of further interest to SQL Server admins is HP’s 
partnership with Microsoft to define a SQL Server 
2008 data-warehouse reference platform based on 
the ProLiant DL580 G5 and HP StorageWorks 50 
Modular Smart Arrays. This platform promises to 
help you mitigate risk and deploy your SQL Server 
2008 data-warehousing solution with confidence and 
predictable results. 


Bronze: 

Dundas Chart for .NET 
Dundas Data Visualization 
www.dundas.com 


SOLE] 
EDITORS’ 
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Why it won: Dundas Chart for .NET is not 
only one of the easiest to use .NET charting solutions 
on the market, but also one of the fastest. Combine 
that with a competitive price and stellar product sup- 
port, and you have a winner. 


WINNERS 


“Using Intel Xeon Quad-Core processors, HP 
ProLiant servers are delivering unprecedented per- 
formance and energy efficiency,” says Tom Kilroy, 
vice president and general manager of Intel’s Digital 
Enterprise Group. “The new four-socket Quad-Core 
Xeon processor systems will provide 16 cores that let 
HP provide customers with superior levels of scal- 
ability and virtualization performance and improve 
their ability to run mission-critical applications.” 

Michael Otey has tested HP ProLiant servers in 
our lab and has found their performance and reliabil- 
ity to be exceptional. “HP’s Integrated Lights-Out 2 
technology makes remote management easy,” Otey 
said. “Plus, the HP DL580 G5’s new Quad-Core 
Xeon processor pushes four-way performance higher 
than ever, making this system very well suited to de- 
manding workloads such as virtualization and appli- 
cation and database serving.” 

—Jason Bovberg 


Silver: 

Dell/Microsoft Business Intelligence 
Data Warehouse 

www.dell.com, www.microsoft.com 
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Why it won: This is a marriage made in 
heaven for SQL Server administrators: Dell rolled out a 
line of servers preconfigured with SQL Server, easing in- 
stallation woes and providing top-notch performance. 


Bronze: 
$ Dell PowerVault MD3000i 
www.dell.com 


ElServer 
EDITORS’ 


BEST 


Why it won: The MD3000i iSCSI SAN ar- 
ray gives you deployment flexibility and 
simple sharing across multiple servers. Busy 
database admins will appreciate the product’s user- 
friendly UI, helpful wizards, and easy operation. 
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Gold: 
RamSan-500 


irl Server 


BEST 
a The latest advancement in stor- 
age technology, solid state disk 
(SSD), provides numerous advantages over older 
storage options. Because no disk rotation is in- 
volved, data access is faster than with DAS, NAS, 
or SANs—SSD access times range from 10 to 15 
microseconds, which is 250 times as fast as hard 
disk drives. SSDs also use less power than other 
storage options, and the lack of moving parts also 
increased reliability. The RamSan-500 provides all 
these advantages and is one of the best SSDs on 
the market today. 


capacity for the best price. 


RaenSar-500 


The RamSan-500 provides up to 2TB of stor- 
age, with as many as 100,000 I/O operations per 
second (IOPS) sustained random reads, 10,000 
IOPS sustained random writes, and 2GBps of 
sustained random read/write bandwidth. Texas 
Memory Systems notes that “compared to a 
similar capacity high-end hard disk-based RAID 
system, the RamSan-500 can deliver in excess of 
16 times the performance while using 50 percent 
less power.” In addition, the RamSan-500 is priced 
lower than competing products, so you get all 
these bells and whistles for a price similar to en- 
terprise RAID storage. Michael Otey says that he 
prefers the RamSan-500 to a similar product from 
BiTMICRO “because of the RamSan’s price and 
capacity.” 

—Lavon Peters 


Silver: 
PS5000E 
Dell * www.equallogic.com 


ElServer 
EDITORS’ 


Why it won: The PS5000E provides 
cost-effective capacity, large scalability, 
and optimal performance for common business ap- 
plications. 


SQL Server Magazine e www.sqlmag.com 


Texas Memory Systems ° www.texmemsys.com 


Why it won: The RamSan-500 makes the jump 
into SSD technology, providing the most storage 
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Your Data is a 
Hidden Treasure, Dive In 


"E 


Bronze: 
E-Disk Altima ATA-133 
BiTMICRO Networks ° www.bitmicro.com 


Why it won: The E-Disk Altima ATA-133 offers huge stor- 
age capacities in a standard disk drive form factor. 
InstantDoc ID 99404 


Data Dynamics 


Y Analysis 


F f $1,499 
Data Dynamics Analysis 
is the first interactive graphical analysis 
component for Windows Forms and ASP.NET 
applications. This tool facilitates real time 
data analysis and promotes the exploration 
of relational and multidimensional data 
sources with simple drag-and-drop operations. 
Developers can embed cutting edge data 
visualization functionality directly into any .NET 
application. Transform raw data into feature-rich 
charts and pivot tables delivering a powerful 


Sou 


and user friendly experience. 


Deliver Business Intelligence 


Ily explore large sets of data 
and obtain new insights that help 
make informed business decisions. 
power Your Users 
Decision makers have direct access 
to information they need to answer 
questions. 


F 
| | Data Dynamics 


4 Reports 
$1,499 


Data Dynamics Reports 

created to handle reports, dashboards, invoices, 
and other documents. The feature rich package 
allows authors to focus on creating documents 
in the intuitive designer instead of writing code. 


Dashboards, calendars, XHTML, mail merge, 
charts, tables, cross-tabs, free-form lists, data 
visualizations, and report parameters are just some 
of the features available right out of the box. 


XHTML and Mail Merge Support 

The Formatted Text report item allows report 
authors to insert XHTML content into their 
reports and it has full support for mail merge 
functionality. 


Data Visualizations 

Graphical representations make it easy to 
understand data. Data visualizations create 
graphics based on your data. Those reading your 
documents can now easily spot the important 
numbers. 


> DATA 
= DYNAMICS 


Comprehensive chart displays reveal trends and 
give users a deep understanding of their data. 


$2,499 bundle price! 


Go Ahead, Show Off 


Excel/Word/PDF Exports 

Take reports with you in any of the 
supported formats: send them out, 
print them out, or just analyze the 
numbers. 


Dashboards 

Dashboard centric controls make it easy 
to create dashboard, use the controls 
anywhere you need to examine 
numbers and trends at a glance. 


www.datadynamics.com 


Tel: (614) 895-3142 
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"Be an Exceptional DBA. 
Use SQL Backup." 


Awards Recognize 


tre BEST -~ Tech-Ed 


2008 DEVELOPERS 


Recognizing exceptional development 
products from Microsoft partners 


he Best of TechEd Awards recognize innova- 

I tive products and services offered by Micro- 

soft partners that exhibited at TechEd 2008 
Developers. 

Our judges consisted of SQL Server Magazine and 
Windows IT Pro editors, who winnowed the 65 submis- 
sions to 13 finalists. On site in Orlando, our editors 
reviewed the finalists to pick winners in five categories, 
including a Breakthrough Product category for the 
most outstanding product of the show. The judges 
evaluated the entries according to three criteria: stra- 
tegic importance to the market, competitive advantage, 
and value to customers. 


Best Developer Tools 
Intercept Studio 4 
AVIcode’s Intercept Studio 4 provides application trouble- 
shooting and health monitoring without requiring any 
changes to application code. An essential tool for trouble- 
shooting and monitoring critical NET applications, 
Intercept Studio 4 captures system activity levels prior to 
application errors, as well as all current system counters, 
code and object values, and user parameters. 

— Michael Otey 


Best Software Components 
and Middleware 
Red Carpet Subscriptions 
In the Software Components and Middleware category, 
our winner is /n software’s Red Carpet Subscriptions. 
This product includes the IPWorks! Internet Toolkit, 
containing TCP/IP communications controls, and inte- 
gration with popular Internet business applications and 
services, including PayPal, credit cards, and FedEx. It 
also includes enterprise adapters for Microsoft BizTalk 
Server and SQL Server Integration Services. 

— Michael Otey 


Best Database 

Development Tools 

Studio for WinForms 

The Best of TechEd in the Database Development Tools 
category is ComponentOne’s Studio for WinForms. 
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Building applications is always easier with a good 
selection of tools, and Studio for WinForms includes 
dozens of ready-made .NET components for devel- 
opers, ranging from calendars, menus, and tabs to 
toolbars and inputs. 

—Jeff James 


Best SharePoint Development 
Tools 
K2 blackpearl 
In the SharePoint Development Tools category, our 
winner is K2 blackpearl. This collection of easy-to-use 
tools lets developers quickly create applications that 
pull together disparate systems and business processes. 
K2 blackpearl can leverage your existing infrastruc- 
ture investment—including SharePoint and Microsoft 
Office Forms Server—to surface the custom applica- 
tions you create. 

—Jeff James 


Breakthrough Product 
CodeRush for Visual Studio.NET , 
Developer Express’s CodeRush for Visual Studio NET 
impressed our judges, and won recognition as the out- 
standing product of TechEd 2008 Developers. Devel- 
oping high-quality code is hard work, and CodeRush 
helps developers optimize their most valuable asset: 
their time. CodeRush lets developers create, reuse, and 
insert complex chunks of code in seconds. 

Jeff James 


Attendees’ Pick 
DXperience 
Developers who attended this year’s TechEd 2008 
Developers voted online for their favorite products. 
After tallying the votes, Developer Express’s DXperi- 
ence emerged as the winner of the Attendees’ Pick 
Award. DXperience is a subscription service that gives 
developers a robust collection of predefined controls 
and tools to help them build .NET applications more 
quickly. [SQL 
—Jeff James 
InstantDoc ID 99516 


by the 
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Editors 
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Master SharePoint 


“= Info-packed 


Hosted by 


WindowsIT Pro 


5, $99) 


WHAT YOU'LL LEARN 


Creating Custom Field Types and Controls for SharePoint Sites 
Leverage SharePoint’s out-of-the-box field types with custom field con- 
trols, and see how to create a custom field type with a custom control 
and value type as well as adding a specialized design time rendering in 
tools such as SharePoint Designer. 


Creating & Debugging Custom Timer Jobs 

in Windows SharePoint Services 

Clean up old sites, send user-created alerts, and disk-quota warnings 
with custom timer jobs, plus learn about the various deployment and 
installation options. 


Leveraging the Content Query Web Part to Its Full Potential 

Go farther with the content query Web Part (COWP) in MOSS 2007 
by configuring the COWP query and additional filtering options and 
customize the styling/rendering of the result set. 


INSTRUCTION LED BY 
Andrew Connell is a Microsoft MVP and an independent 
consultant with a background in content management 
solutions, Web development, and Microsoft's SharePoint 
technologies. 


Check out the SharePoint Mastery 


ros with i . 
Series builtjust io me and Learn more about the speakers, sessions, 
SharePoin 


Microsoft MVP Michael Noel! and how to reserve your seat at: 


WindowsITPro 


Introducing 


Excel Services 
for 


ShhraralPoine 


Not your dad's Excel 


(BI) feature—not of Excel per se, but of Micro- 

soft Office SharePoint Server (MOSS) 2007 
Enterprise Edition—that integrates data-analysis capa- 
bilities into MOSS. Excel Services lets businesses share 
Microsoft Office Excel spreadsheets in a SharePoint 
environment, while centralizing the Excel processing, 
security, and presentation in MOSS. I'll introduce you 
to Excel Services, explain how to configure it and set 
up the necessary data connections, then show you how 
you can use it to do BI reporting and collaboration in 
MOSS 2007. 


E: Services is a key new business intelligence 


Excel Services Basics 

Excel Services consists of three components: Excel 
Calculation Services (ECS), Excel Web Access (EWA), 
and Excel Web Services (EWS). ECS performs all the 
calculations and data refreshes, maintains the user 
sessions, and lets you publish the Excel look and feel 
to an end user who lacks the Excel client. EWA is the 
client side of Excel Services. It consists of the full-page 
report viewer and a Web Part viewer for the Excel 
spreadsheets. EWS is a MOSS 2007 Web service that 
developers can use to develop custom applications for 
Excel Services. 

Excel Services, along with the Business Data 
Catalog (which provides business-data search func- 
tionality) and InfoPath Forms Services make a strong 
case for the additional capital investment required to 
upgrade from the standard edition of MOSS 2007 to 
the Enterprise version. You can find more information 
about licensing and the Enterprise-version features at 
office.microsoft.com/en-us/sharepointserver. 


Excel Services Prerequisites 

We’ll assume that a default Report Center site has 
been created using the Report Center site template, 
and use that as the baseline for our discussion and 
demonstrations. You'll also need SQL Server Analysis 
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MSFTDBProdSamples). 


Services on the SQL server (with all logins configured) 
and you'll need to download and deploy a sample 
AdventureWorks DW database (www.codeplex.com/ 


Configuring Excel Services 

To get started using Excel Services, you'll need to per- 
form some configuration steps. After you've installed, 
configured, and licensed the SharePoint instance on 
your server farm, you must also set up the Shared Ser- 
vice Provider (SSP) in MOSS 2007's Central Adminis- 
tration console. 

To enable the shared service, you need to set up a 
trusted file location for a document library. The loca- 
tion can be a SharePoint document library, Universal 
Naming Convention (UNC) path, or Web site, but it 
must be explicitly trusted before ECS can access it. 

This document library is more processing intensive 
than a typical SharePoint document library. It stores 
documents and calculates and renders as a Web page 
every spreadsheet that’s requested. Because of the 
potential for resource management problems, the site 
collection administrator for the SSP must explicitly 
set up each document library that runs Excel Services 
reports as a trusted file location. He or she also man- 
ages the shared resources on the SharePoint farm. 

You set up a trusted file location in Central Admin- 
istration. Click the Shared Services Provider link listed 
below the Shared Services Administration link as 
Figure 1, page 32, shows. The default name for the first 
provider is SharedServices]. 

Once Shared Services opens, click Trusted file loca- 
tions in the right column under Excel Services Settings. 
Doing so opens the Excel Services Trusted 
File Locations page. Click Add Trusted File 
Location to add your document library to the 
trusted locations. 

In the Location section, add your document 
library URL to the Address field. Using the default 
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Report Center site template, this location would be 
http://sitename/ReportsLibrary. If you include subdi- 
rectories in your document library, be sure to select the 
Children trusted check box. The default Location Type 
is Windows SharePoint Services. 

You can change the default selections in the Session 
Management and Workbook Properties sections. The 


Figure | 


Setting up a trusted 
file location for a 
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default settings are sufficient for our demonstration. In 
the Calculation Behavior section, choose Automatic to 
update all dependent values whenever a value changes. 
There are a few changes to make in the External 
Data section. First, if you're securing a production 
environment, choose Trusted data connections libraries 
only. This selection ensures that each spreadsheet uses 
an approved data connection in a trusted location. (I 
discuss how to set up those trusted data connections 
in a moment.) None is the most secure option because 
it doesn’t allow any connections, but it’s also the least 
usable option. Trusted data connection libraries and 
embedded lets spreadsheet authors embed any data 
connection they want; this is the least secure option. 
Next, clear the Refresh warning enabled check box 
unless you enjoy seeing a warning each time external 
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Figure 2 


Viewing a newly 
created trusted DCL 
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data is refreshed. For our example, we'll leave the 
remaining defaults. Note that the last section, User- 
Defined Functions, lets you allow user-defined func- 
tions (UDFs). Click OK after you’ve completed your 
selections to create the Trusted File Location. 

Now we need to set up the trusted data connection 
library (DCL), where we store the Office Data Connec- 
tion (.odc) file for our Excel Services workbooks to con- 
nect to the SQL Server database. This is much simpler 
than setting up the trusted file location. 

Above the Excel Services Trusted File Locations 
header, click Shared Services Administration: Shared- 
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Services! (Figure 2 shows this link) to return to the 
Shared Services Administration page. Next, click 
Trusted data connection libraries in the Excel Services 
Settings section of the page. 

Click Add Trusted Data Connection Library and 
enter the location of your DCL. Add a description for 
the library and click OK. 


Creating a Data Connection 
Using the Excel Client 

Now that the Excel Services environment is set up and 
ready to go, we need to create and deploy content. First, 
to create dynamic Excel spreadsheets that connect to 
MOSS 2007 and SQL Server 2005, we need to create an 
.odc file to place in the new trusted DCL. Although you 
can create an .odc file from different Office applications, 
we will use the Excel client since we are going to create 
an Excel spreadsheet to publish later. 

Open Microsoft Office Excel 2007. We’ll be working 
with the SQL Server AdventureWorks sample data- 
base; however, you can substitute your own data con- 
nection for this one. Click the Data tab, then From 
Other Sources, then From Data Connection Wizard. 
In the Data Connection Wizard, choose Microsoft 
SQL Server Analysis Services (or the data connection 
of your choice) and click Next. On the Connect to 
Database Server screen, enter your server name and 
click Next. On the Select Database and Table screen, 
select the AdventureWorks DW database and the 
AdventureWorks cube for the specific connection, and 
click Next. 

The Save Data Connection File and Finish screen is 
the key to managing a large volume of .odc files. Fill 
in the information for your data connection and click 
Authentication Settings. 

You have three authentication options to let Excel 
Services connect to your database, retrieve 
live data, and render it in your Excel report. 
Windows Authentication is the default selec- 
$ tion. However, if you use this selection with 
the default SharePoint farm install, it will 
fail. The default SharePoint farm installation 
sets the access method of the global access 
control to trusted subsystem. SQL Server 
doesn’t trust Excel Services to validate the credentials 
requesting the data and won’t return any data. For 
Windows Authentication to work, you need to change 
the global access control to delegation mode (this 
includes a constrained Kerberos setup). This change 
requires running a simple Stsadm command, which 
T'll discuss shortly. 

The second option is single sign-on (SSO). To use 
SSO, you must enable it in Central Administration on 
the SharePoint server and set up an application ID 
for Excel Services. The third option for authentication 
is None. If you’re making a connection to an ODBC 
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datasource or another data system that will use an 
authentication method embedded in the connection 
string, choose None, and the credentials in the string 
will be passed through to the data store. 

Because we want to walk through changing the 
default global access control setting, select Windows 
Authentication and click OK, then Click Finish on 
the Save Data Connection File and Finish screen. We're 
only creating an .odc file to upload into our DCL, so 
on the Import Data window, select Only Create Con- 
nection and click OK to save the new data connection 
file to the My Data Sources folder in your Documents 
folder (My Documents in Windows XP). Now we 
need to upload this file to our library, so that it’s avail- 
able for use. 


Uploading the .odc File 

to the DCL 

Launch your SharePoint site and navigate to the DCL. 
On the library toolbar, click New to upload the .ode 
file. On the Upload Document screen, click Browse. 
Next, in the Choose file menu box, navigate to the 
My Data Sources folder, select the ODC file you just 
created, and click Open. On the Data Connections: 
Filename edit screen, click OK. 

One distinctive characteristic of a DCL is that 
everything you place in it goes to a pending status. This 
constraint forces you to approve the data connections. 
Approve the new data connection file. 

If you consider from a business perspective what 
we just did by uploading this simple file, you can see 
some of the benefits of MOSS 2007 and Excel Services 
integration. We not only made a data connection file 
that will no longer need to be created by every business 
or data analyst in the company, but also placed it in a 
secure library, the security of which you can manage 
at the root level or at the individual file level. 

This approach provides several benefits. First, you 
won't have 20 to 100 of these files floating around 
your network drives. Second, the data connection for 
a particular data store will be consistent throughout 
the organization. Third, if you use item-level security, 
the analyst will see only the data connections he or she 
needs to work with. Not only is this a security measure, 
it’s also a huge benefit to the analyst, who no longer 
has to search through numerous shared drives and files 
with cryptic names. 

By making good use of the connection file’s meta- 
data, a company can greatly enhance the management, 
security, and use of data connection files. Good meta- 
data will let business analysts and other consumers of 
the connection file quickly find the correct file with the 
data connections they need. 

Now that we have an .odc file in our trusted DCL, 
we need to make sure that we use this particular file 
when we use Excel to analyze the data. To do so, we’ll 
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open Excel 2007 and add a data connection using the 
Existing Connections button, which you can find by 
clicking Data from the menu bar, Get External Data 
from the Office ribbon, then Existing Connections 
from the pop-up menu. 

On the Select Data Source screen, which Figure 3 
shows, navigate to the DCL by entering its URL, for 
example http://sitename/sites/reports/Data Connec- 
tions. You'll see the contents of the DCL. 

Select your .odc file and click OK. On the Import 
Data screen, select a PivotTable Report, place the table 
in the Existing Workbook, and click OK. Select a 
couple of items from the PivotTable Field List to create 
a report. For the AdventureWorks example, we'll use 
the Internet Gross Profit and Internet Sales Amount 
for the Values, and we'll select Customer Geography, 
Country as our Row Labels. 

Before we actually publish this report to our 
Reports Library, we want to name the PivotTable. 
Click PivotTable Tools, Options and give the Pivot- 
Table a distinct name. While you're at it, you might 
want to add some visualization to the report. Select 
the values for Internet Gross Profit (cells B3:B8), then 
select Home, Conditional Formatting, Data Bars and 
select the blue data bars. Repeat the same process for 
the Internet Sales Amount (cells C3:C8), but choose 
the green data bars this time. 

One last piece of formatting that will give the report 
a little nicer look is a PivotTable style. Click PivotTable 
Tools, Design and select Pivot Style Medium 16. 

Now that the report is all dressed up, let’s give it 
somewhere to go. Click the Office Button, then Pub- 
lish, Excel Services. In the Save As dialog box, change 
the location to Reports Library. If you don’t have a 
direct link to the Reports Library, paste in its URL 
and press the Tab key to display it. Next, configure 
Excel Services Options. When you click the Excel 
Services Options button, you have two tabs to choose 
from. Choose the first tab, which lets you select which 
workbook parts to publish. 


Figure 
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Selecting a DCL on 
the Select Data Source 


screen 
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_INFRASTRUCTURE LOG 


_DAY 48: This is ridiculous. We’re sitting on a giant 
mountain of information, but we can’t use it in any way 
that drives the business forward. We need info we can 
trust. We need info we can act on. 


_We need to get Gil an oxygen tank. 


_DAY 50: I’m taking back control with IBM. Their Information 
On Demand portfolio can help us optimize our business 

by unlocking the real value of our information. Now we have 
everything we need, including software, services and 
storage, for an end-to-end business solution. We can turn 
trusted business information into actionable insight. 


_Did I use the word “actionable”? Yes, yes, I did. I am Ned. 
Ned who aligns business with IT and uses the word “actionable. 


Watch the Information On Demand video at: 


IBM.COM/TAKEBACKCONTROL/INFO 
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Figure 4 . . ; 
You have three options. You can publish the entire 


Publishing individual workbook, which will render everything in the work- 
named items in book in Excel Services. You can also publish individual 
a workbook sheets from the workbook. This option is useful if 
youre doing calculations based on some proprietary 
business logic that’s located on a separate sheet. You 
can publish the result of that logic in your report 
without exposing the actual calculations that produce 
the report. 

The option we'll select for our example is to publish 
individual named items in the workbook. I named my 
PivotTable ISALES so that when I select Items in the 
Workbook, I'll see that option under All PivotTables. 

Select the PivotTable you created, as Figure 4 shows. 


If we had 

Business Intelligence > Reports Library created a report 
Open > | Update > | A Find that had param- 
eters (e.g., year), 

Ef internet Gross Profit Internet Sales Amount we could publish 


them as param- 


# Australia 685,855.08 


# Canada $829,921.50 eters and make 
+ France $1,086,264.72 $2,644,017.71 them available to 
3 Germany $1,187,370.77 the users of this 
#United Kingdom | $1,390,490.78 report by adding 


# United States them on the 


Parameters tab. 

For our example, 

Finished report we do not need to add any parameters, so simply 
click OK. 

Name your report and click Save. If you're saving 
your report in the Reports Library, you'll be prompted 
to choose a document type. There are two views in the 
Reports Library: One is called Dashboards and is used 
for the dashboard pages and the other is called Current 
Reports and is for reports. For our document, we want 
to choose Current Reports. 

Two things happen when you click OK at this step. 
Obviously, the report will be saved to the library. Not 
so obviously, a copy of the report will be cached for 
display in Excel Services. Because we didn’t clear the 
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Open in Excel Services check box when we saved the 
document, the report will automatically open in Excel 
Services. Figure 5 shows a report in full-screen mode; 
you can also view it as part of a dashboard page. 

There’s one small problem left to deal with. In 
the Excel Services rendering of the PivotTable, click 
Update and select Refresh All Connections. You'll get 
an error stating the data refresh failed. 

Because the Excel Services server was set up in a 
farm, the Access Model defaulted to Trusted Sub- 
system when SharePoint was installed. You must set 
the Access Model to Delegation to allow the data 
in the new workbook to be refreshed. Change the 
Access Model by using the Stsadm command-line 
tool. Open a command prompt and navigate to 
C:\Program Files\Common Files\Microsoft Shared 
\Web server extensions\12\BIN. To change the settings 
we need, enter (on a single line) 


stsadm -o set -ecssecurity -accessmodel delegation 
-ssp Sharedservices1 


The only variable here is the SSP. If you're working 
with a different SSP, be sure to replace sharedservices1 
with that name. Otherwise, you should be able to type 
the command, press Enter, and get a response that says 
Operation completed successfully. 

Stsadm jobs are timer jobs and might take a few 
moments to execute. If you want to force the jobs in 
the timer queue to run immediately, you can type the 
following command: 


stsadm -o execadmsvcjobs 


Important: After the job has executed, you must 
also run lisreset.exe, as follows: 


C:/>iisreset Inoforce 


Now the access model will be in Delegation mode, 
and your report should execute and refresh. 


A Robust Solution 

Microsoft says it invests more than $700 million each 
year in research and development in its Office pro- 
grams, servers, and services. There can be no doubt 
that the company spent a good amount of that effort 
on Excel Services. As a management tool, it’s a robust, 
centrally managed document system with individual 
file-level security. As a delivery tool, it’s a high-end 
Excel rendering engine that maintains a high level of 
visual fidelity and doesn’t require Excel 2007. As a 
development platform, the new EWS provides a reus- 
able framework that provides a foundation for custom 
development with Excel data. 

All of these things combine to make the first gen- 
eration Excel “server.” It’s a tool many developers will 
agree was far too long in coming. SQL 
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the release of SQL Server 7.0, 
icrosoft SQL Server became 

a true enterprise-level database 
management system (DBMS). But 
the evolution didn’t stop there. SQL 
Server has come a long way in the past 10 years. 
This white paper provides an overview of SQL Server 
enhancements during this evolutionary journey, 
covering the SQL Server suite of services and tools 
for both the relational database engine and business 
intelligence (BI). 


Organized historically, this white paper covers the 
most important features from version to version, 
showing how the product has matured over time to 
satisfy ever growing business needs. The paper starts 
with SQL Server 7.0, moves to SQL Server 2000 and 
2005, then ends with a discussion of SQL Server 
2008, which will be available this year. Although this 
paper focuses on the ongoing improvement process, 
it also mentions some unfortunate features—including 
some that have been or will be abandoned. 


SQL Server 7.0: Sphinx 

Our historical look at SQL Server begins with Version 
7.0, also known by the code name Sphinx. This version 
was revolutionary in many ways. Microsoft rewrote the 
storage engine from scratch, including a new locking 
mechanism, and provided completely new tools. SQL 
Server 7.0 was the first version that seriously targeted 
the enterprise market, providing both performance and 
scalability. And SQL Server 7.0 was the first version 

of the product that included OLAP functionality for 

free within the suite, which changed the OLAP market 
forever. Starting with SQL Server 7.0, OLAP became 
affordable for small and medium businesses, and was 
no longer limited solely to large enterprises. 


A New Database Engine 

The code base for SQL Server originated in Sybase 
SQL Server. But with SQL Server 7.0, Microsoft moved 
away from Sybase code, rewriting the database engine 
from scratch. Microsoft developers did an excellent 
job; the SQL Server 7.0 storage engine remains the 
foundation for newer versions of the SQL Server 
storage engine. 


Microsoft developers increased the basic storage 
unit for data, the page, from 2K to 8K. The same 
page size is still used in SQL Server 2008. SQL 
Server 7.0 was also the first version that dynamically 
managed memory and files on disk. Although some 
IT professionals thought these management features 
would make the database administrator (DBA) role 


obsolete, they instead freed up DBAs to focus on more 
meaningful tasks, making the role more valuable than 
ever, especially for medium and large databases. 
However, with SQL Server 7.0’s new self-management 
capabilities, small companies no longer needed 
extensive database management knowledge in-house 
to operate SQL Server. 


SQL Server 7.0 was the first version of the product 
available on desktop computers through the free 
Microsoft Desktop Edition (MSDE). This edition— 
called SQL Server Express in later versions—was 
and still is useful together with merge replication to 
support disconnected users. Merge replication, also 
new in SQL Server 7.0, lets disconnected workers 
such as sales representatives travel with their laptops, 
insert orders, and update customer data, then merge 
their changes with changes on the central production 
server when they reconnect. Merge replication worked 
in all editions of SQL Server 7.0, including MSDE. In 
addition to merge replication, SQL Server 7.0 also 
provided snapshot and transactional replication to 
support the need for distributed, “scale out” data 
scenarios among multiple databases and locations. 


Besides the rewritten storage engine, Microsoft 
developers also introduced an entirely new set of 
database engine tools for SQL Server 7.0. The 

main administrative tool was Enterprise Manager, a 
Microsoft Management Console (MMC) snap-in. For 
performance tuning and troubleshooting, SQL Server 
Profiler provided a rich user interface for setting up and 
analyzing server traces. Using server traces, you can 
capture the commands SQL Server is processing as 
well as SQL Server’ s responses to those commands. 
You can capture errors, warnings, execution times, 
disk I/O, memory and CPU consumption, and much 
more. SQL Server 7.0 also lets you then use another 
tool—the Index Tuning Wizard (ITW)—to quickly get 
reasonable recommendations for indexes based on 
this realtime usage data from Profiler. The ITW worked 
well as long as the sample of captured queries was 
representative of your overall workload. 


The main new SQL Server 7.0 tool for developers was 
SQL Server Query Analyzer. Although it had a small 
footprint, Query Analyzer was powerful and popular 
among database developers; many of us still miss 
this tool, which was replaced in SQL Server 2005 

by functionality in the new SQL Server Management 
Studio (SSMS). Query Analyzer was handy for ad hoc 
queries and for troubleshooting; it was the first tool 
capable of graphically showing a query’s execution 
plan, making the plan intuitive to understand. Figure 1 


shows an example of an execution plan in SQL Server 
7.0 Query Analyzer. 


Among the many database engine enhancements in 
SQL Server 7.0, developers especially appreciated 
the new locking schema. With the new version, the 
lowest level of locking granularity became the row 
level instead of the page level. Locking escalation 
was dynamic, so you didn’t need to manually set 

the lock escalation threshold. In addition, accessing 
remote data sources from SQL Server was easier than 
ever before thanks to linked servers. A linked server, 
in short, is a named connection string that uses an 
OLE DB provider. In SQL Server 7.0, you could refer 
to remote data from a Transact-SQL (T-SQL) query 
simply by providing the four-part object name: server, 
database, schema, and object. SQL Server 7.0 was 
also the first version that let you use full-text searches 
on database columns; this capability let you search for 
words or phrases against character data. 


With Windows integrated security fully implemented 
in SQL Server 7.0, DBAs could stop worrying about 
brute-force attacks on SQL logins if they used 

only trusted connections and disabled the mixed 
authentication mode. However, when using mixed 
authentication mode, enabled SQL logins were still 
exposed to brute-force attacks because SQL Server 
did not implement any password or account policies, 
like account lockout policy, for SQL logins. 
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The dramatic improvements in SQL Server 7.0’s 
storage engine and tools led to better and more 
controllable performance and provided appropriate 
scalability for most business needs in the late ‘90s. 
However, DBAs still had a hard time achieving high 
availability. For example, to correctly set up SQL 
Server in a cluster, you had to work through a 70-page 
“How to install SQL Server 7.0 Enterprise Edition on 
Microsoft Cluster Server” white paper, then check 
Microsoft's “Order of Installation for SQL Server 7.0 
Clustering Setup” Knowledge Base article for updated 
information. 


OLAP Revolution 

Although the new SQL Server 7.0 database engine 
was significant, the IT community expected it. 
Customers demand that Microsoft continuously deliver 
database improvements, and the company must 

also try to stay ahead of or abreadst of competitors, 
who are also always working to move their products 
forward. The real revolution in SQL Server 7.0 
occurred in BI. Although many different Online 
Analytical Processing (OLAP) servers and client tools 
were on the market at the time, they all had a common 
failing: They were very expensive. The high prices 
were primarily related to the target buyers of OLAP 
solutions—decision makers, who also usually control 
the money. By including OLAP Services in the SQL 
Server 7.0 suite for free, Microsoft changed the OLAP 
market forever, bringing OLAP to the masses. 
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OLAP lets end users change the view of displayed 
data or the report they’re looking at in realtime, without 
needing to ask for help or for a report update from IT 
support or developers. Two main pillars enable realtime 
analysis: simple, predictable schema, which enables 
building simple and intuitive client tools to navigate 
around, sort, and analyze the data; and OLAP server 
performance, which enables the needed speedy data 
access. OLAP uses the well-known star schema, 

which features two types of tables: fact tables and 
dimensions. Fact tables hold things you are measuring 
(i.e., measures), and dimensions consist of attributes 
you are using to break down your measures (i.e., the 
elements that let you drill down until you find interesting 
information). For example, a Sales fact table might 
hold such measures as sales quantity and sales 
amount, while the Customer dimension might have 
such attributes as country, region, city, and customer. 
You start analyzing from the top, checking the total 
sales, then drill down to individual countries, regions, 
cities, and customers. OLAP achieves lightning analysis 
speed through precomputed aggregates. So, for 
example, from detailed data in your source system, the 
OLAP server calculates totals across cities, regions, 
and countries in advance instead of processing every 
user request at the time of the request. 


SQL Server 7.0 provided several OLAP tools: OLAP 
Services, which was the OLAP server; Analysis 

Manager, a tool for developing OLAP databases and 
cubes for administering OLAP databases; and client 
tools. Microsoft extended Office Excel to serve as an 
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OLAP Services client tool, and third-party tools started 
appearing soon after. Probably the most popular 
client tool was ProClarity, a rich client from a company 
called Knosys; the company later changed its name to 
ProClarity, and in 2007, Microsoft acquired it. Figure 

2 shows a typical star schema being built in OLAP 
Manager. 


One star schema typically covers one business 
subject, such as sales, finance, or warehouse. 

You connect multiple star schemas through shared 
dimensions. And you almost always have a time 
dimension as well because nearly all organizations 
need to perform analysis over time. Before building 
any OLAP cubes, you need to consolidate data from 
multiple sources into multiple star schemas. Typically, 
this consolidation is performed in a relational database 
called a data warehouse. A data warehouse consists 
of one or more star schemas, with merged, cleansed, 
and historical data. From a data warehouse, you build 
your OLAP cubes. In SQL Server 7.0, you couldn't 
build an OLAP cube from scratch; the schema was 
inherited from a relational data warehouse. In addition, 
a single cube was limited to a single fact table (i.e., 

a single star schema). Although it was possible to 
compare data from two fact tables--for example, from 
actual sales and the sales plan—it wasn’t simple. 


To keep your data warehouse current, you must 
regularly extract data from sources, transform it into 
your data warehouse schema, and load it to the data 
warehouse. This regular process is called Extract- 
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Figure 3: Installing SQL Server 2000 on a Virtual Server 
for Failover Clustering 


Transform-Load (ETL). You can write your own ETL 
application from scratch, but you can also use a tool 
that helps you build such an application. With SQL 
Server 7.0, Microsoft shipped a new tool called Data 
Transformation Services (DTS) that let you build ETL 
applications called packages. DTS packages were task- 
oriented. In one task, for example, you would extract the 
data, do the transformations, and load the transformed 
data in a staging table. Then, you would use another 
task to re-read the same data for another purpose. 
Thus, DTS was actually more ELT than ETL tool. 


With DTS, the only way to read data once, do multiple 
transformations in memory, and write the data, was to 
use the Data Pump task. Inside the Data Pump task, 
you could transform data by using an ActiveX scripting 
language such as VB Script. Nevertheless, with the 
new SQL Server database engine, OLAP Services, and 
DTS, SQL Server 7.0 provided a very powerful platform 
for satisfying business needs for data and information. 


SQL Server 2000: Shiloh 

Although not as revolutionary as SQL Server 7.0, 
SQL Server 2000, code-named Shiloh, nevertheless 
delivered substantial improvements that made the 
product even more useful in companies of all sizes. 
SQL Server 2000 database engine enhancements 
included expansion of the RDBMS to portable 
devices, supporting multiple instances of SQL 

Server on a single computer, new and expanded 
integrity checking mechanisms such as an expanded 
Foreign Key constraint and Instead Of triggers, better 
performance, easier replication management, and a 
new mass notifications functionality called Notification 


om aa) 


Services. And with its continued BI 
improvements, SQL Server 2000 
battled for leadership in a market that 
it had entered only one version earlier. 


RDBMS Everywhere: 

From Portable Devices to 
Datacenters 

With the expansion of Web 
applications and different production 
systems automatically collecting more 
and more data, the size of databases 
grew dramatically in the late ‘90s. 
SQL Server 2000 was prepared for 
the challenges. You could install SQL 
Server 2000 on the most powerful 
Microsoft operating system of that time, 
Windows 2000 DataCenter Server, 
with 64-bit architecture support. You could scale out 
your database across multiple servers. And you could 
split a table’s rows across multiple tables, which could 
reside in different databases on different servers, and 
then union all of the rows together through distributed 
partitioned views (DPVs). SQL Server 2000 also 
featured many other performance improvements , such 
as letting you materialize views and computed columns 
(i.e., store the result sets in the database) by indexing 
them. 


As many companies started Web and application 
hosting, they needed many SQL Server instances. 
Installing them on dozens of computers and 
purchasing the associated licenses would have simply 
been too costly. But SQL Server 2000 let you install 
up to 16 instances on a single computer, with all the 
licenses covered by a single Enterprise Edition (EE) 
license. And on the lower end, SQL Server Compact 
Edition (CE) worked on CE devices and supported 
merge replication, extending disconnected applications 
to nearly any device. 


Besides peak performance, the other indispensible 
requirement for mission-critical applications, especially 
Web applications, is high availability. As one of its 
high-availability features, SQL Server 2000 supported 
up to four-node clusters for failover clustering. The 
setup procedure was completely cluster aware—no 
more following a lengthy white paper to properly install 
SQL Server in a cluster. Figure 3 shows SQL Server 
2000’s cluster-aware installation wizard. 


For database developers, user-defined functions 
(UDFs) were a valuable addition to T-SQL in SQL 
Server 2000. In addition, the product recognized XML’s 


growing popularity as a data-interchange format. For 
the many applications that needed data from SQL 
queries in XML format, Microsoft extended the SELECT 
statement in SQL Server 2000 with the FOR XML 
clause. Instead of a relational table, the result of such 

a statement was XML. Of course, you couldn't use this 
clause if you needed your data in relational format (e.g., 
in a view definition). However, being able to retrieve the 
data in XML format let you skip a step in transforming 
relational data from tabular format to XML format at the 
middle tier or client application, thus enabling faster 
development and better performing applications. 


SQL Server 2000 improved data integrity from the 
referential integrity point of view by extending the 
Foreign Key constraint to cascading options. A 
foreign key prevents orphaned children in one-to- 
many relationships between entities. For example, 

a foreign key prevents orders without customers. 
SQL Server achieves this integrity through four 
rules: preventing inserts without a parent existing on 
the child side, updating the parent key on the child 
side to a value that does not exist on the child side, 
preventing deletion of a parent with children, and 
preventing update of the parent key on the parent 
side if the parent has children. Although a Foreign 
Key constraint in SQL Server 7.0 implemented these 
four rules, the implementation led to a lot of coding 

if you needed to delete a parent with children. For 
example, you had to delete the children first, probably 
in a stored procedure. SQL Server 2000 added two 
more possibilities for the last two referential rules on 
the parent side. With the new CASCADE option, you 
could specify deletion of the children if their parent was 
deleted or automatic propagation of the parent key 
change to all children. 


SQL Server 2000 also introduced Instead Of triggers. 
Before this version, triggers in SQL Server could fire 
only after the transaction. With Instead Of triggers, you 
can intercept the original data modification command, 
check for data integrity issues, and reapply a correct 
command. In addition, you can intercept the data 
modification commands directed to a non-updateable 
view and send modification commands to underlying 
tables, thus making the view updateable. 


Replication in the new version added support for 
automatic management of identity ranges in merge 
replication scenarios. In SQL Server 7.0, snapshot 
and transactional replication already allowed updates 
on the replica database (i.e., on the subscribers). 
However, these updates were executed through a 
distributed transaction synchronously on the source 


(i.e., on the publisher) and on the subscriber. SQL 
Server 2000 added support for asynchronously 
updating subscribers by using queues. You could 
implement queues through the Microsoft Message 
Queue (MSMQ) service or through SQL Server tables. 


A couple of years after SQL Server 2000's initial 
release, Microsoft added SQL Server Notification 
Services (SSNS) to the version as a free, 

downloaded component for Standard or Enterprise 
customers. SSNS was more a development tool 

than an application, letting developers quickly build 
applications that needed to perform mass notifications 
of subscribers. 


Battle for BI Leadership 

On the BI side, SQL Server 2000 became a serious 
player. The version considerably improved OLAP 
cubes, and you could now define security settings on 
the lowest possible granularity level-on members of 
a dimension and even on cells of a cube. SQL Server 
2000 added an important aggregate function, the 
Distinct Count function. Distinct count is important for 
market basket analysis, for example, which is based 
on the theory that if a customer buys a certain group 
of items, he or she is more (or less) likely to buy 
another group of items. In the new RDBMS version, 
dimensions could support hierarchical relations if you 
used the adjacency model in your relational tables. In 
the adjacency model, you use an entity key, a parent 
entity key, and a foreign key between the entity and 
parent entity keys to model a hierarchy. Examples of 
hierarchies are employees and their managers, bills 
of material, and so on. To use such a hierarchy for 

a report, you have to expand the adjacency model 

in a tree. In SQL Server 2000, this could be done 
automatically by so-called parent-child dimensions. 
You could use parent-child dimensions for ragged and 
unbalanced dimensions as well. Figure 4 shows a 
hierarchy modeled in an adjacency model. 
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Figure 4: The Adjacency Model 


OLAP analysis isn’t the only kind of analysis 
organizations need to perform. With OLAP, for 
example, you have to define the model in advance 
inside OLAP cubes through star schema. Then, you 
can do model-driven analysis, drilling down through 
one attribute at a time. To create a view of fact data 
over a specific combination of dimension attributes to 
gain more insight than a view over a single attribute, 
you have to work with a client tool for a while, 
especially to find the most meaningful combination 
of dimension attributes. Wouldn't it be nice to have a 
tool that could find the most meaningful combination 
for you automatically? That functionality is what a 
technology called data mining provides. With data 
mining, you analyze your data by using advanced 
mathematical methods to find meaningful patterns and 
rules (i.e., a meaningful model). Thus, data mining 
provides data-driven analysis. 


SQL Server 2000 added support for two of the most 
popular data mining algorithms: the Decision Trees 
and Clustering algorithms. And because the OLAP 
service wasn’t just for OLAP analysis anymore, 
Microsoft renamed it Analysis Services and renamed 
OLAP Manager to Analysis Manager. 


Data mining wasn’t mature in Analysis Services 2000. It 
was Microsoft's first foray into the data mining world and 
couldn’t compare with older products. However, it paved 
the road for the future. Competitive tools were very 
expensive. And you could imagine the same revolution 
coming in the data mining market as happened in the 
OLAP market with SQL Server 7.0—data mining would 
sooner or later be available for the masses. 


As noted earlier, ETL applications can be complex, and 
the better tools you have, the easier it is to develop 
such applications. In the ETL space, SQL Server 2000 
brought significant enhancements to DTS. Standing 
out among the many new DTS tasks was the Dynamic 
Properties task, which let you read connection 
settings, table names, and just about anything else for 
your DTS package from an .ini file, a global variable, 
an environment variable, a query, a constant, or a data 
file. The Data Pump task became a Multiphase Data 
Pump, letting you customize the data pump at various 
phases of its operation and add functionality such as: 
e row-level restartability 
e individual handling of insert or transformation errors, 
such as constraint violation errors 
e customization of initialization or termination steps 


Using the same release process as for SSNS, a 
couple of years after SQL Server 2000’s initial release, 


Microsoft added reporting functionality to the product 
through a free downloadable component called SQL 
Server Reporting Services (SSRS). As with SSNS, 
SSRS was free for users with a Standard or Enterprise 
Edition license. SSRS, still a valuable component 

in SQL Server 2008, is a platform for creating and 
managing reports for relational as well as BI needs. 
Not all users need advanced BI capabilities such 

as OLAP and data mining, and SSRS adds another 
way to fulfill business needs for gathering useful 
information from a sea of data. However, many users 
didn’t appreciate that the only development tool for 
SSRS in SQL Server 2000 was Visual Studio (VS)— 
and that VS wasn’t shipped with SSRS. If you didn’t 
have a valid VS license in place, SSRS wasn’t exactly 
“free” for you. 


A less fortunate part of the BI suite in SQL Server 2000 
was English Query. With English Query, you could 

write applications that accepted users’ input entered as 
English language questions and then transformed the 
questions into database queries. English Query’s life 
span was very short; it existed only in SQL Server 2000, 
never making it to the next version of the product. 


SQL Server 2005: Yukon 

SQL Server 2005, code-named Yukon, was the next 
release of the database system, and it was a major 
one. Microsoft rewrote most of the BI suite from scratch, 
for example, which is the main reason there are five 
years between Shiloh and Yukon. Among the many new 
features and enhancements in SQL Server 2005 are 
Common Language Runtime (CLR) and XML support in 
the database engine, new transaction-isolation levels, 
rewritten tools, improved performance and reliability, 
and security that meets modern needs. From the BI 
perspective, the new version delivered a rewritten 
Analysis Services with mature data mining capabilities, 
a completely new ETL tool called Integration Services, 
and improvements in Reporting Services. 


Overcoming T-SQL Limitations 

Many DBAs’ first reaction when they started to explore 
the novelties in Yukon was fear. SQL Server 2005 

is the first version of the product not limited solely to 
the T-SQL language. Yukon adds support for CLR 
languages and for XQuery. Using CLR languages 
such as Visual Basic .NET and Visual C#, you can 
write stored procedures, UDFs, triggers, and two new 
objects that you cannot write in T-SQL-user-defined 
aggregate functions and user-defined data types. 


DBAs were especially afraid of the CLR language 
support, airing their concerns about security and 


performance. T-SQL is a limited language—limited 
to data operations inside SQL Server only. With CLR 
code, developers can do nearly anything from inside 
SQL Server, thus introducing a whole new set of 
security issues. In addition, DBAs were concerned 
that with CLR code, developers would move from 
efficient set-based operations in SQL Server to row- 
based processing and thus introduce a lot of new 
performance issues in database applications. 


However, Microsoft had not forgotten about security. 
DBAs were quickly comforted by the fact that they do 
not have to deal with security in detail. For CLR code, 
SQL Server 2005 introduced three permission sets: 
SAFE, EXTERNAL_ACCESS, and UNSAFE. The 
SAFE set, the recommended permission set, limits 
CLR code to operations that T-SQL can do. With the 
EXTERNAL_ACCESS set, code from SQL Server 
can access external resources such as the file system 
and network resources. With the UNSAFE set, CLR 
code can do nearly anything, including directly writing 
to memory, so it can endanger SQL Server itself. Of 
course, you should have a good reason to use any 
permission set except SAFE and be especially careful 
with the UNSAFE set. In addition, CLR is disabled 

by default; a DBA must enable it for an instance by 
using the sp_configure system stored procedure. 
Concerns about poor performance of CLR code also 
disappeared over time; DBAs realized that a developer 
can write good or bad code in any language. 


Besides adding the capability of creating CLR data 
types, SQL Server 2005 introduced some new 
system data types. Probably the most important 
among them is the XML data type. You can finally 
store XML documents where they should have been 
from the start: in a relational database. Because 
XML is a large data type, letting you store up to 
2GB of data, customers need a way to retrieve and 
update just part of it. This is where XQuery comes 
in. XQuery is a query language for traversing XML 
nodes, finding elements and attributes, and even 
looping through nodes at a selected level. SQL Server 
2005 implements XQuery support through XML data 
type methods. The XML data type has methods to 
retrieve and update parts of XML data, and XQuery 
expressions are parameters of those methods. The 
methods are: 
e query(), which returns part of the XML data 
in XML format 
e value(), which returns a scalar value of an 
element or an attribute of an element 
e exist(), which returns 1 if an element or an 
attribute exists in an XML instance, 0 if an 
element or an attribute doesn’t exist, and null 
if the XML data type instance contains null 
e modify(), which lets you insert an element or 
an attribute, delete an element or an attribute, or 
update the value of an element or an attribute 
e nodes(), which lets you shred an XML data type 
instance into relational data 
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Microsoft wasn’t just concerned about CLR code 
security in SQL Server 2005. Yukon contains many 
other security enhancements, including properly 
implementing schemas—as namespaces for database 
objects. You no longer have to use object owner to 
group objects. All we can say is—finally! Schemas 
should have been introduced years earlier. You 

can now also set permissions on a schema, and 

the permissions are inherited on objects inside the 
schema, making the job of setting up and maintaining 
permissions much easier. 


Other security enhancements include SQL logins 
that are finally protected; SQL Server 2005 enforces 
Windows account and password policies for SQL 
logins. In addition, triggers are no longer limited to 
data modification operations; a DBA or developer can 
write data definition language (DDL) triggers to, for 
example, prevent table modifications in a production 
database. Data encryption is also now possible with 
new encryption functions and complete key and 
certificate support. However, using encryption in SQL 
Server 2005 means upgrading your applications. 
Yukon adds many new detailed permissions as 

well, and you can overcome problems with broken 
ownership chains by specifying the execution context 
for programmable objects. Figure 5 show the complete 
key and encryption hierarchy. 


From the manageability point of view, the difference 
between Yukon and previous versions is immense. 
With new catalog and dynamic management views 
and functions, a DBA has detailed insight into SQL 
Server operations, state, and resource usage in real 
time, with T-SQL queries. In addition, a DBA can use 
a dedicated admin connection, which runs in a higher 
thread priority, to quickly kill a runaway session. 

You can now gather and save execution plans in 
XML format, which is simple to consume within an 
application. This ability simplifies the viewing of a large 
number of plans and lets you easily use the plans as 
input for support. 


The SQL Server community also needed some time 
to really appreciate another Yukon feature: completely 
rewritten tools. The primary DBA tool is now SQL 
Server Management Studio (SSMS), a centralized 
tool for managing the database engine, SQL Server 
Analysis Services (SSAS), SQL Server Reporting 
Services (SSRS), SQL Server Integration Services 
(SSIS), and mobile edition servers. SSMS also lets 
you write queries. SSMS replaces Enterprise Manager, 
Analysis Manager, Query Analyzer, and other tools. 
Because SSMS is based on VS, you can easily add 


versioning support to it by using Team System Server 
or Visual SourceSafe. 


Besides CLR and XQuery support, developers also 
received new T-SQL elements in SQL Server 2005. 
New operators—including PIVOT, UNPIVOT, and 
APPLY—can help abbreviate your code. And the 
version boasts new ranking functions, including 
ROW_NUMBER(), which have topped wish lists for 
years. Yukon introduces Common Table Expressions 
(CTEs) as another way of using subqueries; with them, 
you can introduce subqueries in advance, in the WITH 
clause, before the outer or main SELECT statement. 
In addition, with CTEs, you can expand hierarchies 
modeled as the adjacency model. 


Asynchronous applications are simpler and 

more reliable than ever with SQL Server Service 
Broker, a new message queuing system inside 

SQL Server 2005. An application can subscribe to 
query notifications; with query notifications, SQL 
Server informs an application that the cached data 

is outdated. And to enable applications in which 
readers do not block writers and to mitigate moving 
applications from Oracle, Yukon adds two new 
transaction-isolation levels: Snapshot and Read 
Committed Snapshot. Thus, SQL Server is no longer 
limited to the pessimistic locking approach only; these 
two levels implement an optimistic approach, storing 
old versions of rows before updates in the tempdb 
system database, then driving SELECTs through these 
old versions. 


Yukon delivers new performance features as well. You 
can now physically partition a table horizontally while 
still referring to it logically in queries by using a single 
table name. This capability makes partitioning inside 

a database much easier than with partitioned views. 
Another new ability in SQL Server 2005 is related to 
database snapshots, which are read-only snapshots of 
a production database at a specific time point. You can 
now drive reading through database snapshots and, 
with them, have another way of implementing reading 
that doesn’t block writing. And the Index Tuning Wizard 
(ITW) has matured into the Database Tuning Advisor, 
which gives better optimization recommendations, 
including suggestions for using table partitioning. 


SQL Server 2005 also provides new and improved 
high availability technologies. You can implement 

up to eight nodes in a cluster configuration. You 

can also now perform indexing online, meaning that 
applications can use a table while it is being indexed. 
The online indexing feature uses the same row 


versioning framework used to implement snapshot 
transaction isolation levels. Last but not least in the 
area of new high availability features in Yukon is 
database mirroring, which is essentially an inexpensive 
alternative to clustering. Database mirroring maintains 
two copies of a single database that must reside on 
different instances of SQL Server, which can reside on 
computers in different locations. One server instance, 
the principal server, is the production server, while the 
other server instance, the mirror server, acts as a hot 
or warm standby server. Automatic failover is possible 
if a third server instance, the witness server, is present. 


Rewritten BI Suite 

SQL Server 2000 gained a well established position 
among major players in the BI market. However, 
Microsoft decided to rewrite almost all products in the 
SQL Server 2005 BI suite, including the development 
tools. The new Business Intelligence Development 
Studio (BIDS) is just another name for VS. The tool, 
which includes templates for all BI projects, now ships 
with SQL Server, providing a separate tool from SSMS 
for developing BI applications. Version management 
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of BI projects is simple, using the same version 
management as any VS project. Customers that do not 
have a valid VS license no longer have to purchase it 
just for SSRS projects, as they had to in SQL Server 
2000. In addition, SQL Server 2005 extends Profiler, 
letting you trace SSAS as well as the database engine. 


Although as one of the newer BI additions, SSRS 
didn’t need major changes, the reporting component 
lacked a simple tool that let advanced end users 
author reports. End users can’t use VS, and they don’t 
know how to write database queries. So Microsoft 
offered a solution. In SSRS 2005, DBAs can create 
report models, semantic descriptions of database 
metadata that serve as an intermediate layer between 
report author and data. Report authors can then create 
reports by using business entities and attributes, and 
SSRS uses the report models to transform the report 
requests into database queries. End users create ad 
hoc reports by using Report Builder, a new, lightweight 
and simple-to-use report authoring tool. In addition to 
the report the end user creates, SSRS automatically 
generates click-through reports, letting consumers 
follow navigation paths based on foreign keys that 
exist within the report model. 


As noted earlier, in SQL Server 2005, SQL Server 
Integration Services (SSIS) replaces DTS. The 
difference is enormous. SSIS is an actual ETL tool. 
Besides control flow and tasks, it features a separate 
data flow engine. Inside a data flow, which you create 
by using the Data Flow task in the control flow, you can 
connect to different sources, read the data, store it in 
memory in tabular format, work on this data through 
multiple transformations, and finally load it to one or 
more destinations. The Data Flow task is actually 

Data Pump on steroids. SSIS includes both basic and 
advanced transformations that you can use to add 

a lot of intelligence to your ETL package. The Fuzzy 
Lookup transformation, for example, can match rows 
from different sources based on similarity of character 
columns, and the Fuzzy Grouping transformation is 
handy for de-duplicating rows based on similarity of 
strings. The Data Mining transformation lets you add 
predictions from a data mining model to your data flow 
and later filter the data based on those predictions. And 
Term Extraction and Term Lookup transformations are 
useful together with SSAS data mining for text mining. 
The Slowly Changing Dimension transformation isn’t 
actually a transformation; it is a wizard that generates all 
the transformations you need for maintaining changes in 
dimension data over time. Inside a data flow, a package 
need not fail on erroneous rows when executed; 
instead, you can define separate error flows and either 
correct or log erroneous rows. Figure 6 shows the 
relationship between control flow and data flow. 


Besides tasks and precedence constraints, control 
flow also includes containers. Two containers, For 
Loop and For Each Loop, allow looping of tasks. You 
can base precedence constraints not only on success, 
failure, or completion of a task, but you can now 
include custom logical expressions to define the flow 
exactly as you want it. These features together with 
package variables, event handlers, extensive logging, 
restartability, external package configurations, and 
improved security make SSIS a high performance 
enterprise ETL tool. 


In SSAS 2005, the term OLAP is replaced with Unified 
Dimensional Model (UDM). UDM isn’t just a marketing 
term; metadata—the schema of OLAP cubes—is 
enhanced so that it is richer than relational schema 

in a data warehouse. Therefore, you do not need 

to first create a data warehouse, then inherit star 
schemas in OLAP cubes; you can start building your 
dimensional model from either the SSAS or relational 
engine side. Thus, SQL Server 2005 delivers true 
unified dimensional modeling. Also in the new version, 
a single OLAP cube can include multiple fact tables. 
Because a cube can now represent a complete data 
warehouse, SSAS 2005 includes additional objects 
called Perspectives. Perspectives are UDM views 
that let users see only the part of the cubes they are 
interested in. In the new version, you can pivot data 
and drill down using any attribute of a dimension, 

not only attributes that are part of a hierarchy. For 
example, you aren’t limited to drilling down only 
through the path country-region-city-customer; you 
can also drill down from country to customer account 
manager, then to size of company, and so on. Inside 
the UDM, you can define translations for data and 
metadata. Besides storing calculations centrally in 
your UDM, you can also store Key Performance 
Indicators (KPIs). By centralizing KPIs and attribute 
properties, such as format strings and presentation 
information, you can maintain a single version of truth 
in the enterprise. No matter where your UDM data is 
retrieved from and with which client tool, it is always 
presented in a consistent way, thus unifying the 
perception to end users. Note that you can also install 
multiple instances of SSAS in SQL Server 2005. And 
the number of supported instances on a single box has 
risen from 16 for SQL Server to 50 for SQL Server and 
50 for SSAS. Finally, you aren’t limited to a scheduled 
refresh of OLAP data if you store it ina UDM; UDM 
supports proactive caching. 


SSAS 2005's data mining features underwent an even 
bigger overhaul. Data mining in version 2005 isn’t just 
a teaser; it is a mature product, integrated with other 

SQL Server components. SSAS ships with all the most 


popular data mining algorithms: 

e Association Rules 

e Decision Trees, which includes Regression Trees 

e Naive Bayes 

e Neural Networks 

e Clustering 

e Sequence Clustering 

e Linear Regression 

e Logistic Regression 

e Time Series, a forecasting Auto-Regression Trees 
(ART) algorithm 


Although not part of the SQL Server suite, Microsoft 
Office Excel 2007 now serves as a first-class OLAP 
client—and with freely downloadable data mining add- 
ins, as a data mining client as well. In addition, Excel 
2007 with data mining add-ins can use SSAS data 
mining algorithms to analyze spreadsheet data. 


SQL Server 2008: Katmai 

The SQL Server story continues this year with SQL 
Server 2008, code-named Katmai. While perhaps 

not as revolutionary as SQL Server 2005, Katmai 
nevertheless is an important step forward in satisfying 
ever growing data-management needs. New system 
data types support spatial applications and sparse 
data and improve date and time handling. The 
MERGE statement, change tracking, and change data 
capture (CDC) simplify data warehouse maintenance. 
Policy-based management, resource control, and 
performance data collecting enable DBAs to work 
more efficiently. SSIS 2008’s new Data Profiling task 
lets you check data quality before you actually start 
your ETL process. And the entire BI suite—including 
SSIS, SSAS, and SSRS—gets a performance boost. 


Supreme Database Engine 

Having CLR support in the database engine enabled 
Microsoft to use it to ship additional data types in 

SQL Server 2008. Katmai supports spatial data and 
spatial applications with Geometry and Geography 
data types, and maintaining hierarchies is simplified 
with the Hierarchyld data type. For large objects, you 
no longer have two mutually exclusive options: storing 
them in the database or in the file system; you can 
now use the new Filestream storage attribute to store 
unstructured data, such as documents and images, on 
the file system while working with the attribute as you 
do with regular large object database data types. The 
benefit of using Filestream data is that you don’t inflate 
your database files with large objects, but you can 
administer large objects together with other database 
objects, including performing backups and setting up 
security. In SQL Server 2008, full-text indexes are 
integrated in the database as well, making them easier 


to maintain and providing better performance for full- 
text searches. 


SQL Server 2008 also delivers one of the most 
requested features for years: new Date and Time 

data types that let you store data and time separately. 
Additionally, the new Datetime2 type has a larger date 
range and a larger default fractional precision than the 
old Datetime type, as well as optional user-specified 
precision. The new Datetimeoffset type offers another 
way to handle date and time, defining a date combined 
with a time of a day that has time zone awareness and 
is based on a 24-hour clock. When you create a table, 
you can also define some columns as sparse columns. 
Sparse columns are ordinary columns that have 
optimized storage for null values. Sparse columns 
reduce the need for separate subtype tables for the 
not-applicable attribute. 


Pure database developers will be happy with the 
extended T-SQL language features in SQL Server 
2008. Stored procedures and UDFs can accept table- 
valued parameters. The new MERGE statement, also 
known as UPSERT, can do multiple data modifications 
at a time. Based on a source table, you can update 
matched rows in the destination table, insert rows that 
do not exist in the destination table yet, and delete 
rows from the destination table that do not exist in 

the source table anymore. This is especially useful 

for maintaining a data warehouse. Two other options 
target data warehouse scenarios as well: Change 
Tracking (CT) and Change Data Capture (CDC). With 
CT, you can track whether there were changes in 
source tables, then efficiently use MERGE statements 
to implement the changes in your data warehouse. 
However, what happens if you don’t have any attribute 
useful for row matching in the source and destination 
tables? This could occur if the primary key changed 

in the source table. In that case, you could use CDC, 
which captures all the changes in additional system 
tables. Because the capturing feature uses data from 
the transaction log asynchronously, it has much lower 
performance impact on the source server than if you 
implemented this feature using data modification 
language (DML) triggers. 


For application developers, SQL Server 2008 supports 
the Entity Framework and Language Integrated Query 
(LINQ). Both application and database developers 

will appreciate the version’s CLR enhancements, 
especially large user-defined data types, which are 

no longer limited to 8K, and multi-parameter user- 
defined aggregate functions. Database encryption 

is transparent for applications in Katmai, meaning 


you should be able to use it without application 
modifications. 


On the DBA side, there are also plenty of reasons to 
look forward to Katmai. Take a look at Figure 7, which 
shows some interesting new folders in SSMS. The new 
Policy Management folder and subfolders are where 
you implement Policy-Based Management (PBM); the 
Data Collection folder lets you set up performance 
data collection; and the Resource Governor folder 

and subfolders let you control resources used by SQL 
Server operations. 


PBM is a policy-based management framework 

for the SQL Server database engine. It ensures 
compliance with policies for system configuration, 
enables preventing and monitoring changes to the 
system, and reduces total cost of ownership by 
simplifying administration tasks. Resource Governor 
is a new technology that lets you manage workload 
and resources by specifying limits on resource 
consumption by incoming requests. This feature is 
useful, for example, if you need to run a huge query 
during peak hours and you do not want the query 

to consume all the resources, such as memory and 
CPU. DBAs will also find SQL Server 2008’s automatic 
performance and other management data collection 
features valuable. You can collect this data in a 
Management Data Warehouse (MDW), which you can 
create—together with the required SSIS packages and 
SSRS reports for analyzing the data—with a couple 

of mouse clicks in SSMS. All these features together 
comprise the new Performance Studio. 


In addition, SQL Server 2008 lets you optimize 
database file space consumption with data and backup 
compression. And new filtered indexes let you index 
only interesting subsets of rows, thus making indexes 
smaller and more effective. 


Note that SQL Server Notification Services (SSNS) 
doesn’t exist in SQL Server 2008. Apparently Microsoft 
realized that not many customers were using the 
component. However, users of existing SSNS 
implementations aren’t going to be happy with this 
decision. 


Mature BI 

The BI suite in SQL Server 2008 continues to mature, 
with SSIS, SSAS, and SSRS seeing advanced 
features, usability enhancements, and performance 
improvements. 

Probably the most important new feature in SSIS 2008 
is the Data Profiling task. Before you transfer source 
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data to your data warehouse, you should always 

perform some quality checks. You can use the Data 

Profiling task to measure the following: 

e Column length distribution, which reports all the 
distinct lengths of string values in the selected 
column and the percentage of rows in the table that 
each length represents 

e Column null ratio, which reports the percentage of 
null values in the selected column 

e Column pattern, which reports a set of regular 
expressions that cover the specified percentage of 
values in a string column 

e Column statistics, which reports statistics, such 
as minimum, maximum, average, and standard 
deviation for numeric columns and minimum and 
maximum for datetime columns 

e Column value distribution, which reports all the 
distinct values in the selected column and the 
percentage of rows in the table that each value 
represents; can also report values that represent 
more than a specified percentage of rows in the table 

e Candidate keys, an option that reports whether a 
column or set of columns is a key, or an approximate 
key, for the selected table (i.e., measures uniqueness 
of a column or set of columns) 
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Figure 7: New Management Folders in SSMS 


e Functional dependency, which reports the 
extent to which the values in one column (the 
dependent column) depend on the values 
in another column or set of columns (the 
determinant column) 

e Value inclusion, which computes the overlap 
in the values between two columns or sets of 
columns; this profile can determine whether 
a column or set of columns is appropriate to 
serve as a foreign key between the selected 
tables, or if all keys in a subtype are included 
in the pertaining supertype 


The main focus of improvements in SSAS 
2008 is performance. Performance is key in 
all phases of the SSAS database life cycle, 
starting with design. OLAP models can be 
complex. And best practices and performance 
tips aren’t generally well known, so designing 
the models correctly isn’t easy. In SSAS 2008, 
performance best practices are embedded in 
the design process. When you design a UDM 
database in BIDS, you get more than 40 best 
practices warnings integrated into realtime 
designer checks. You see blue squiggly lines 
and receive build-time warnings if your model 
isn’t in accordance with performance best 
practices. In addition, the Dimension Designer has a 
new Attribute Relationship Designer. The Cube Wizard 
doesn't create all possible attribute hierarchies by 
default, so you don’t automatically get aggregates 
over attributes that you never use for drilling down, 
such as customer address and phone. The Attribute 
Relationship Designer also gives you more control 
over the design of aggregates. When a UDM is 
deployed, a DBA can monitor performance through 
new Dynamic Management Views (DMVs). SSAS also 
exposes server resource information as a cube that 
you can use to perform resource analysis. 


SSAS 2008 also boasts performance improvements 
for cubes in production. Calculating expressions on 
sparse cubes is optimized so much that you can 

feel the magnitude of improvement compared to 
SSAS 2005—and many OLAP cubes are sparsely 
populated. If you write back to cubes—for example, 
for a planning and budgeting application—in previous 
versions, write-back was done in the relational data 
warehouse; reading relational data was magnitudes 
slower than reading OLAP data. In SQL Server 2008, 
multidimensional OLAP storage (MOLAP), a variant 
where you store everything (including metadata, 
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Figure 8: Excel 2007 as SSAS 2008 Data Mining Client 


aggregates, and data) in an SSAS database, isn’t 
read-only anymore. You use MOLAP write-back, thus 
gaining a lot in performance. SSAS 2008 also supports 
scalable read-only shared databases; a database 

can use SAN storage and is shared between multiple 
Analysis Services instances. Backup speed is also 
improved. 


SQL Server 2008 brings a couple of data mining 
improvements as well. Time Series analysis now 
supports the Auto-Regressive Integrated Moving 
Average (ARIMA) algorithm in addition to the ART 
algorithm. ART is better in short-term forecasting, and 
ARIMA performs better for long-term forecasts. You 
can use a mix of ART and ARIMA forecasts in a single 
mining model. Microsoft will also upgrade Office 2007 
Data Mining Add-Ins to support SSAS 2008. In Figure 
8, you can see Excel 2007 as a data mining client tool 
for SSAS 2008. 


In SSRS 2008, | want to point out two important 
features in particular. The first one is the Tablix data 
region. A Tablix data region combines features of both 
a table and a matrix, letting you organize data both by 


column and row groups. Some features enabled with 
the Tablix data region include the possibility of having 
stepped columns with crosstab reports—for example, 
customer country and region can share the same 
column; using side-by-side crosstab sections—for 
example, product category and color; and having 
independent group aggregates with crosstab reports. 
The other exciting SSRS 2008 feature is the new 
authoring tool. Report Designer is now a standalone 
application with an Office 2007 look and feel. It is easier 
to use than VS, yet it doesn’t lack any functionality. 


Work in Progress 

From Version 7.0 to Version 2008, SQL Server has 
provided and continues to provide a reliable, scalable, 
and secure database platform that satisfies growing 
business needs. However, it is much more than a 
relational database management system. With all its 
database engine and BI components, SQL Server 
delivers everything you need to store, maintain, 
integrate, and analyze your data. It is a mature, 
market-leading product. However, the work is never 
finished, and there is still room for improvement. 

In future releases, the first one expected in 2011, | 


would like to see support for temporal applications 
built into the database engine. And the OVER clause 
of the T-SQL SELECT statement would be even 
more useful if it fully implemented all ANSI-standard 
subclauses. From the tools perspective, | really miss 
a good database modeling tool and would like to see 
one included in the SQL Server suite. | would also 
like to see a setup procedure that includes simple 
rollback of the installation. In the BI suite, we need 
more tools for checking data quality and getting 

an overview of the data. | would like to see more 
statistical algorithms implemented in SSAS, maybe 
through language enhancements or through system 
stored procedures. Based on the enhancements that 
Microsoft has delivered in the last four SQL Server 
versions, | am confident that future versions will 

give us even more advanced, efficient, and easy- 
to-use ways to meet our business needs for storing, 
managing, and analyzing data. 
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Hierarchy 


Move subtrees and convert 
parent-child representations of hierarchies 


Server 2008’s new data type, 
called HIERARCHYID, pro- 
vides powerful new capabili- 


ties. For example, you can use HIERARCHYID to 
represent nodes in a hierarchy and to query the hier- 
archy, as I demonstrated in last month’s column. (See 
“HierarchyID,” July 2008, InstantDoc ID _99036.) 
Another capability you can use is to move a subtree 
from one parent location in the hierarchy to another 
by using the GetReparentedValue method. In addi- 
tion, you can convert a parent-child representation 
of the hierarchy to a representation that uses the new 
HIERARCHYID data type. I'll walk through these 
capabilities in this month’s column, using the same 
Employees table that I used in last month’s column. If 
youre following along, you'll want to run the code in 
Listing 1, page_38, to create the Employees table and 
populate it with sample data. 


Moving a Subtree 

When I showed how to insert new nodes into the hier- 
archy last month, I explained how to use the GetDescen- 
dant method to calculate a HIERARCHYID value for 
a new node under an existing node. You might need to 
handle other types of changes within hierarchies as well. 
For example, you might need to move a whole subtree of 
employees from one manager to another (i.e., move an 
employee and all of his or her subordinates). 

To achieve this task you can use the GetReparented- 
Value method of the HIERARCHYID data type. You 
invoke this method on the HIERARCHYID value 
of the node you want to reparent (call it @existing_ 
node_hid), and provide as inputs the value of the old 
parent (call it @old_parent_hid) and the value of the 
new parent (@new_mgr_hid). Here’s the syntax of the 
GetReparentedValue method: 


@existing_node_hid.GetReparentedValue 
(@old_parent_hid, @new_parent_hid) 
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Note that this method doesn’t overwrite or change the 
value in @existing_node_hid. Instead, it returns a new 
HIERARCHYID value. You must update the target 
node’s HIERARCHYID value with the new one. Logi- 
cally, the GetReparented Value method does something 
very simple—tt substitutes the part of the existing node’s 
path that represents the old parent’s path with the new 
parent’s path. If the path of the existing node is /x/a/, 
the path of the old parent is /x/, and the path of the new 
parent is /y/, the GetReparentedValue method would 
return /y/a/. For example, if the path of the existing node 
is /1/2/1/, the path of the old parent is /1/2/, and the path 
of the new parent is /2/1/1/4/, the GetReparentedValue 
method would return /2/1/1/4/1/. 

If the target parent already has children, the Get- 
ReparentedValue method won't necessarily produce 
a unique value. If you reparent node /x/a/ from old 
parent /x/ to new parent /y/, and /y/ already has a child 
/ylal, you will get a duplicate value. The trick is that 
you don’t use the GetReparentedValue method to 
move a single node from one parent to another; for this 
task you can simply use the GetDescendant 
method, and produce a completely new value. 
You use the GetReparentedValue method 
when you need to move a whole subtree in con- 
junction with the GetDescendant method. 

Suppose you need to move the whole subtree of 
employee /x/a/ from old parent /x/ to new parent /y/ 
that already has children. You achieve this with the 
following steps: 

1. Use the GetDescendant method to produce a 
completely new HIERARCHYID value under /y/ 
(call it /y/b/). 

2. Update the HIERARCHYID value of all nodes 
in the subtree of /x/a/ (including self), to the return 
value of node.GetReparented Value(/x/a/, /y/b/). 
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itzik@ solidqualitylearning.com), a mentor at 


Solid Quality Learning, teaches, lectures, and 
consults internationally. He manages the Israeli 
SQL Server Users Group, is a SQL Server MVP, 
and is the author of the Inside Microsoft 
SQL Server 2005: T-SQL series (Microsoft 

Press, 2006). 


ORE on the WEB 


Download the listings at 
InstantDoc ID 99369. 


Because /y/b/ is a completely new HIERARCHYID 
value under the target parent, /y/b/ has no existing 
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LISTING I: Code to Create and Populate the 
Employees Table 


-- Script to Create the Employees Table 

SET NOCOUNT ON; 

USE tempdb; 

GO 

IF OBJECT_IDC'dbo.Employees', 
DROP TABLE dbo.Employees; 

GO 

CREATE TABLE dbo.Employees 

K 


"U') IS NOT NULL 


empid INT NOT NULL, 

hid HIERARCHYID NOT NULL, 

lvl AS hid.GetLevel() PERSISTED, 

empname VARCHAR(25) NOT NULL, 

salary MONEY NOT NULL, 

CONSTRAINT PK_Employees PRIMARY KEY NONCLUSTERED(empid) 
Vs 


CREATE UNIQUE CLUSTERED INDEX idx_depth_first ON dbo.Employees (hid) ; 


CREATE UNIQUE INDEX idx_breadth_first ON dbo.Employees(lvl, hid); 


GO 


-- Script to Create Stored Procedure usp_AddEmp 
IF OBJECT_IDC'dbo.usp_AddEmp', 'P') IS NOT NULL 
DROP PROC dbo.usp_AddEmp; 


(e0) 

CREATE PROC dbo.usp_AddEmp 
@empid AS INT, 
@mgrid AS INT, 


@empname AS VARCHAR(25), 
@salary AS MONEY 


AS 

DECLARE 
@hid AS HIERARCHYID, 
@mgr_hid AS HIERARCHYID, 


@last_child_hid AS HIERARCHYID; 
BEGIN TRAN 


IF @mgrid IS NULL 

SET @hid = HIERARCHYID: :GetRoot() ; 
ELSE 
BEGIN 


SET @mgr_hid = (SELECT hid FROM dbo.Employees WITH C(UPDLOCK) 


WHERE empid = @mgrid); 
SET @last_child_hid = 
(SELECT MAX(hid) FROM dbo.Employees 
WHERE hid.GetAncestor(1) = @mgr_hid); 
SET @hid = @mgr_hid.GetDescendant(@last_child_hid, NULL); 
END 


INSERT INTO dbo.Employees(empid, hid, empname, salary) 
VALUES(@empid, @hid, @empname, @salary); 


COMMIT TRAN 
GO 


-- Code to Populate Table Employees 
EXEC dbo.usp_AddEmp 


@empid = 1, @mgrid = NULL, Gempname = ‘David’ , @salary = 
EXEC dbo.usp_AddEmp 

@empid = 2, @mgrid = 1, Gempname = ‘Eitan’ , @salary = 
EXEC dbo.usp_AddEmp 

@empid = 3, @mgrid = 1, Gempname = ‘Ina’ , @salary = 
EXEC dbo.usp_AddEmp 

@empid = 4, @mgrid = 2, @empname = ‘Seraph’ , @salary = 
EXEC dbo.usp_AddEmp 

@empid = 5, @mgrid = 2, @empname = ‘Jiru’ , @salary = 
EXEC dbo.usp_AddEmp 

@empid = 6, @mgrid = 2, @empname = ‘Steve’ , @salary = 
EXEC dbo.usp_AddEmp 

@empid = 7, @mgrid = 3, @empname = ‘Aaron’ , @salary = 
EXEC dbo.usp_AddEmp 

@empid = 8, @mgrid = 5, @empname = ‘Lilach’ , @salary = 
EXEC dbo.usp_AddEmp 

@empid = 9, @mgrid = 7, @empname = ‘Rita’ , G@salary = 
EXEC dbo.usp_AddEmp 

@empid = 10, @mgrid = 5, @empname = ‘Sean’ , @salary = 
EXEC dbo.usp_AddEmp 

@empid = 11, @mgrid = 7, G@empname = ‘Gabriel’, @salary = 
EXEC dbo.usp_AddEmp 

@empid = 12, @mgrid = 9, @empname = ‘Emilia’ , @salary = 
EXEC dbo.usp_AddEmp 

@empid = 13, @mgrid = 9, @empname = ‘Michael’, @salary = 
EXEC dbo.usp_AddEmp 

Gempid = 14, @mgrid = 9, @empname = ‘Didi’ , @salary = 
GO 
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children; hence, moving the subtree doesn’t cause con- 
flicting HIERARCHYID values. 

Run the code in Listing 2 to create the usp_ 
Reparent stored procedure that implements this logic 
to move a subtree. The code in the procedure first 
opens a transaction. Within the transaction, the code 
retrieves the hid value of the new manager (@new_ 
mgr_hid) and specifies the UPDLOCK hint to obtain 
an UPDATE lock on the row. Only one transaction 
can hold an UPDATE lock at a time; once obtained, 
this lock is kept until the end of the transaction. This 
way, if simultaneous invocations of usp_Reparent 
and usp_AddEmp make requests to create a new 
HIERARCHYID value under the same target man- 
ager, those requests won’t produce duplicate values, but 
instead will be queued. 

The code retrieves the hid value of the old manager 
(@old_root). Next, the code uses the GetDescendant 
method to produce a new HIERARCHYID 
value under the new manager (@new_root). Then, 
the GetReparentedValue method updates the 
HIERARCHYID values of the employee that is moved 
and of all descendants of that employee. Finally, the 
code commits the transaction, releasing all locks. 

To demonstrate using the usp_Reparent procedure, 
run the code in Listing 3. The code first queries the 
Employees table, producing the output shown in Table 1. 
The code then invokes the usp_Reparent procedure to 
reparent the subtree of employee 5 (Jiru) under man- 
ager 9 (Rita). Finally, the code queries the Employees 


LISTING 2: Script to Create the 
usp_Reparent Procedure 


IF OBJECT_IDC('dbo.usp_Reparent') IS NOT NULL 
DROP PROC dbo.usp_Reparent; 

GO 

CREATE PROC dbo.usp_Reparent 
@empid AS INT, 
@new_mgrid AS INT 

AS 


DECLARE 
@old_root AS HIERARCHYID, 
@new_root AS HIERARCHYID, 
@new_mgr_hid AS HIERARCHYID; 


BEGIN TRAN 


SET @new_mgr_hid = (SELECT hid FROM dbo. 
Employees WITH CUPDLOCK) 
WHERE empid = @new_mgrid) ; 
SET @old_root = (SELECT hid FROM dbo.Employees 
WHERE empid = G@empid); 


-- First, get a new hid for employee that moves 
SET @new_root = @new_mgr_hid.GetDescendant 
CCSELECT MAX(hid) 
FROM dbo. Employees 
WHERE hid.GetAncestor(1) = @new_mgr_hid), 
NULL) ; 


-- Next, reparent all descendants of employee 
that moves 
UPDATE dbo.Employees 

SET hid = hid.GetReparentedValue(@old_root, @ 
new_root) 
WHERE hid.IsDescendantOf(@old_root) = 1; 


COMMIT TRAN 
GO 
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LISTING 3: Code to Test the usp_Reparent Procedure 


SELECT empid, REPLICATE(' | 
FROM dbo.Employees 
ORDER BY hid; 


BEGIN TRAN 


EXEC dbo.usp_Reparent 
@empid = 5, 
@new_mgrid = 9; 
SELECT empid, REPLICATE(' | ', 


FROM dbo. Employees 
ORDER BY hid; 


ROLLBACK TRAN 
GO 


table again to present the hierarchy after the subtree 
has been moved. Table 2 shows the output of this 
query. 


Converting Parent-Child to 
HIERARCHYID 
Suppose you have an existing hierarchy represented as 
an adjacency list with parent-child relationships, and 
you need to convert it to a representation based on the 
new HIERARCHYID data type. To achieve this task, 
first run the code in Listing 4 and Listing 5, page 40. 
The code in Listing 4 creates and populates the source 
EmployeesOld table, where the parent-child relation- 
ships are reflected by the mgrid-empid attributes. The 
code in Listing 5 creates the target EmployeesNew table, 
where the converted hierarchy of employees will be 
stored. The hid attribute will hold the HIERARCHYID 
value for each employee. 

To perform the conversion, I take the following steps: 
1. Define a regular common table expression (CTE— 


TABLE |: Employees Hierarchy Before 
Moving Subtree of Jiru under Rita 


empid empname path 

1 David / 

2 Eitan /1/ 

4 | Seraph /1/1/ 

5 | Jiru /1/2/ 

8 | | Lilach /1/2/1/ 
10 | | Sean /1/2/2/ 
6 | Steve /1/3/ 

3 Ina /2/ 

7 | Aaron /2/1/ 

9 | | Rita /2//1/ 
12 | | | Emilia /2/1/1/1/ 
13 | | | Michael /2/1/1/2/ 
14 | | | Didi /2/1/1/3/ 
1i | | Gabriel /2/1/2/ 
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lvl) + empname AS empname, hid.ToString() AS path 


call it EmpsRN) that calculates a row number for 


', lvl) + empname AS empname, hid.ToString() AS path 


Q Editors Note 


We’ve included all the 

listings and tables for this 
article in print rather than 
only online. Which do you 
prefer? Send your feedback 


to Ipeters @ penton.com. 


each employee; the row number is partitioned by 
the manager (mgrid) and ordered by the attributes 
that you want to determine order among siblings 
(say, empid). 

2. Define a recursive CTE (call it EmpPaths) that 
queries EmpsRN and builds a character path for 
each employee. The anchor member will query the 
root employee (CEO), and assign ‘/’ as the path. 
The recursive member will query the direct subor- 
dinates of the previous level of managers in each 
iteration, then concatenate to the manager’s path 
the current employee’s row number and a slash. 

3. Query the EmpPaths table and convert the path 
that was produced for each employee to the 
HIERARCHYID data type, and insert the result 
rows to the EmployeesNew table. 


The complete solution that converts the old hier- 


archy to the new one is shown in Listing 6, page 40. 


TABLE 2: Employees Hierarchy After 
Moving Subtree 


empid empname path 

il David / 

2 | Eitan /V/ 

4 | | Seraph /A/1/ 

6 | | Steve /1/3/ 

3 | Ina /2/ 

7 | | Aaron /2/1/ 

9 | | | Rita /2/1/1/ 

12 | 1 | | Emilia 2A 
13 | | | | Michael /2/1/1/2/ 
14 | 1 I I Didi /2/1/1/3/ 
5 111 WJiru /2/A/1/4/ 
8 | 11 I I Lilach /2/1/1/4/1/ 
10 | 1 1 1 I Sean /2/1/1/4/2/ 
11 | | | Gabriel /2/1/2/ 
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LISTING 4: Code to Create and Populate 


the Table EmployeesOld 


IF OBJECT_IDC'dbo.EmployeesOld') IS NOT NULL 
DROP TABLE dbo.EmployeesOld; 


(e0) 
CREATE TABLE dbo.EmployeesOld 
K 
empid INT PRIMARY KEY, 
mgrid INT NULL REFERENCES dbo.EmployeesOld, 


empname VARCHAR(25) NOT NULL, 
salary MONEY NOT NULL 


CREATE UNIQUE INDEX idx_unc_mgrid_empid ON dbo.EmployeesOld(mgrid, empid); 


INSERT INTO dbo.EmployeesOld(empid, mgrid, empname, salary) 


VALUES(1, NULL, 'David', $10000.99) ; 


INSERT INTO dbo.EmployeesOld(empid, mgrid, empname, salary) 


VALUES(2, 1, 'Eitan', $700@.9); 


INSERT INTO dbo.EmployeesOld(empid, mgrid, empname, salary) 


VALUES(3, 1, 'Ina', $7500.90); 


INSERT INTO dbo.EmployeesOld(empid, mgrid, empname, salary) 


VALUES(4, 2, 'Seraph', $5900.00) ; 


INSERT INTO dbo.EmployeesOld(empid, mgrid, empname, salary) 


VALUES(5, 2, 'Jiru', $5500.09); 


INSERT INTO dbo.EmployeesOld(empid, mgrid, empname, salary) 


VALUES(6, 2, 'Steve', $4500.99); 


INSERT INTO dbo.EmployeesOld(empid, mgrid, empname, salary) 


VALUES(7, 3, ‘Aaron', $5900.00); 


INSERT INTO dbo.EmployeesOld(empid, mgrid, empname, salary) 


VALUES(8, 5, 'Lilach', $3580.98); 


INSERT INTO dbo.EmployeesOld(empid, mgrid, empname, salary) 


VALUES(9, 7, 'Rita', $3000.09); 


INSERT INTO dbo.EmployeesOld(empid, mgrid, empname, salary) 


VALUES(19, 5, ‘Sean', $3000.00); 
INSERT 

VALUES(11, 7, 'Gabriel', $3009.00); 
INSERT 

VALUES(12, 9, 'Emilia' , $2000.00); 
INSERT 

VALUES(13, 9, 'Michael', $2000.98); 
INSERT 

VALUES(14, 9, 'Didi', $1598.90); 
GO 


NTO dbo.EmployeesOld(empid, mgrid, empname, salary) 
NTO dbo.EmployeesOld(empid, mgrid, empname, salary) 
NTO dbo.EmployeesOld(empid, mgrid, empname, salary) 


NTO dbo.EmployeesOld(empid, mgrid, empname, salary) 


TABLE 3: Converted Hierarchy 


empname 
David 

| Eitan 

| | Seraph 

| | Jiru 

| | | Lilach 

| | | Sean 

| | Steve 

| Ina 

| | Aaron 

| | | Rita 

| | | | Emilia 
| | | | Michael 
| 1 I I Didi 

| | | Gabriel 
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path 


/1/ 

A/A 
/1/2/ 
/1/2/1/ 
/1/2/2/ 
/1/3/ 

/2/ 

/2/\/ 
/2/\/1/ 
PAAA 
/2/1/1/2/ 
/2/1/1/3/ 
12/1/2/ 


LISTING 5: Code to Create the 
Table EmployeesNew 


IF OBJECT_ID('dbo.EmployeesNew') IS NOT NULL 
DROP TABLE dbo.EmployeesNew; 

ie) 

CREATE TABLE dbo.EmployeesNew 

K 
empid INT NOT NULL PRIMARY KEY, 
hid HIERARCHYID NOT NULL, 
lvl AS hid.GetLevel() PERSISTED, 
empname VARCHAR(25) NOT NULL, 
salary MONEY NOT NULL 

5 

(e0) 


LISTING 6: Code Converting 
Parent-Child to HierarchyID 


WITH EmpsRN 
AS 
K 
SELECT empid, mgrid, empname, salary, 
ROW_NUMBER() OVER(PARTITION BY mgrid 
ORDER BY empid) AS rn 
FROM dbo.EmployeesOld 
), 
EmpPaths AS 
G 
SELECT empid, mgrid, empname, salary, 
CAST('/' AS VARCHAR(9@8)) AS cpath 
FROM dbo.EmployeesOld 
WHERE mgrid IS NULL 


UNION ALL 


SELECT C.empid, C.mgrid, C.empname, C.salary, 
CAST(cpath + CAST(C.rn AS VARCHAR(29)) + '/' 
AS VARCHAR (988) ) 
FROM EmpPaths AS P 
JOIN EmpsRN AS C 
ON C.mgrid = P.empid 
) 
INSERT INTO dbo.EmployeesNew(empid, empname, salary, hid) 
SELECT empid, empname, salary, 
CAST(cpath AS HIERARCHYID) AS hid 
FROM EmpPaths; 


After running the code in Listing 6, query the new 
hierarchy as follows: 


SELECT REPLICATE(‘ | ‘, lvl) + empname AS 
empname,hid.ToStringQ AS path 

FROM dbo.EmployeesNew 

ORDER BY hid; 


You'll get the output shown in Table 3, where you can 
see the logical paths that were originally constructed 
from the row numbers of the employees in the manage- 
ment chain leading to each employee. 


Practice, Practice, Practice 
The new HIERARCHY ID data type provides a native 
way to handle hierarchies, with behavior and function- 
ality encapsulated in the type and exposed in the form 
of methods. The purpose of having a native type is to 
simplify maintaining and querying hierarchies, but as is 
typical with new tools, the HIERARCHYID data type 
requires some getting used to. Requests for a subgraph, 
path, presentation, reparenting, and so on simply look 
different when the hierarchy is represented with the 
HIERARCHYID data type. Only after you spend 
enough time practicing with a new tool can you truly 
appreciate whether it improves your experience. EX 
InstantDoc ID 99369 
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Digging GSMS 


Get familiar with some powertul, 
lesser-known features of SOL Server 


Management Studio 


Server 2005 introduced a new 

tool for managing servers and 

databases: SQL Server Man- 
agement Studio (SSMS). SSMS has more features than 
its predecessors in earlier SQL Server versions—and 
some of those can take a little while to figure out. To 
help ease the learning curve, I'll provide some tips on 
how to use SSMS effectively for tasks such as querying 
and reporting, focusing on features that are less obvious 
to new users. TIl base this discussion on SSMS in SQL 
Server 2005 SP2. If you're using an earlier version, 
some features, captions, or menu options might be 
slightly different. 


Exploring Object Explorer 

When you open SSMS, most often you'll either use the 
Object Explorer window to work with servers and data- 
bases, or you'll execute queries. Object Explorer fea- 
tures a welcome enhancement: asynchronous loading 
of object lists. Rather than making you wait while a list 
of objects is retrieved from the server, SSMS gives you 
a visual notification and lets you proceed with other 
tasks in the meantime. 

You might have noticed that when you right-click 
most objects in Object Explorer, you get many options 
for scripting the object (e.g., Create, Alter, Drop). You 
can output the script into a new window, the Clip- 
board, or a file. What you might not know is that you 
can drag and drop almost all objects from the Explorer 
window into a query window. In most cases, SSMS 
outputs the object’s name into the query window. But 
for the Columns node, SSMS writes out the names of 
all columns in the table, potentially saving you a lot of 
typing. The table node offers many other productivity 
enhancements, such as improved scripting for INSERT, 
UPDATE, DELETE, and many other statements, 
as well as the ability to view the data or design the 
table. The table node’s Edit function is somewhat mis- 
named; the Edit function pretty much just generates a 
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CREATE script for re-creating the table. 

Besides working with databases, SSMS also lets 
you connect to and work with SQL Server Reporting 
Services (SSRS), SQL Server Integration Services, SQL 
Server Analysis Services, and also SQL Server Com- 
pact Edition. This functionality, however, is limited 
mainly to administration and management of deployed 
packages, reports, or cubes. If you want to build and 
debug these types of objects, you have to use Business 
Intelligence Development Studio. Despite the fancy 
name, it’s really Microsoft Visual Studio 2005 equipped 
with SQL Server-telated projects. 


Working with Queries 

One of my preferred options when working with que- 
ries is to have query results open in a full 
window. This option was somewhat easier to 
find in Query Analyzer than it is in SSMS. 
If you want to maximize the screen showing 
the results, go to Tools, Options, Query 
Results, SQL Server, Results to Grid or Results to Text 
and select Display Results in a separate tab. Now the 
query opens in a separate subtab, but it’s still grouped 
together under the same tab with the Editor and Mes- 
sages windows. You can use the F6 keyboard shortcut 
to switch between the query and the results. 

The query results window has two neat improve- 
ments in the grid mode that aren’t in Query Analyzer. 
First, when you highlight cells in the results and paste 
them into Microsoft Excel or another application, 
SSMS automatically inserts column names in the first 
row, saving you the work of typing them explicitly. 
Note that this feature isn’t enabled by default. You can 
enable it in the SSMS settings; to do so, open Tools, 
Options, Query Results, SQL Server, Results to Grid 
and set the Include column headers when copying or 
saving the results option on. 

The other not-so-obvious functionality is that you 
can now rearrange the order of columns in the results 
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by simply dragging and dropping the column headers. 
This capability is especially useful when you're working 
with the results returned by a stored procedure and 
you want to change the order of the columns so you 
can examine the data more easily. You can now do so 
without rewriting the query in the stored procedure. 

In SSMS, Microsoft rolled up the functionality of 
SQL Server Enterprise Manager and Query Analyzer 
into one tool. The most common argument I’ve heard 
from DBAs against SSMS is that it can be overkill if 
all you need to do is quickly open a query window and 
run some queries. Although I love SSMS—and have 
been using it for more than two years—occasionally 
TIl still open Query Analyzer (via a keyboard shortcut 
saved on the desktop) when I need to do a querying 
task quickly. 

If you fall into this camp, I might have good news 
for you. At the 2007 Professional Association for SQL 


SUL Server Management Studie Properties 


Server (PASS) Commu- 
nity Summit, I was in a 
session given by amember 
of the SQL Server tools 
team who mentioned that 
Microsoft was seriously 
looking into providing 
the ability to load SSMS 


Figure | 
Disabling the splash 
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in a lightweight mode 
without Object Explorer 
and other windows. 
This implementation— 
if it happens—should 
improve the experience 
for those who still desire 
the speed and simplicity 
of Query Analyzer. In the 
meantime, though, here are my recommendations for 
making SSMS more Query Analyzer—like: 

* Close unnecessary windows. SSMS comes with many 
windows docked on the right or on the left side— 
for example, Properties, Template Explorer, and 
Toolbox. You can eliminate some of these distrac- 
tions and recover useful screen space by closing the 
windows you don’t use often and setting the ones 
you use to Auto Hide (by clicking the Auto Hide 
pushpin so that it points to the left). I typically keep 
only the Registered Servers, Object Explorer, and 
Solution Explorer loaded and set to Auto Hide. 

* Disable splash screen loading. In my tests, disabling 
the splash screen can cut the load time for SSMS in 
half. The load time is still hard to predict because 
sometimes for no apparent reason SSMS takes 10 to 
15 seconds to load, but most of the time it loads in 
a second or two when the splash screen is disabled. 
You can disable the splash screen by adding the 
-nosplash command-line parameter to the SSMS 
shortcut, as Figure 1 shows. 


e Disable certificate check. Many Microsoft pro- 
grams run a check over the Internet to verify the 
publisher every time you start them. Disabling this 
option can boost the load time. You can disable this 
check using Microsoft Internet Explorer; to do so, 
go to Tools, Internet Options, click the Advanced 
tab, and scroll down to the Security section. Next, 
uncheck the Check for publisher's certificate revoca- 
tion option. You should disable the certificate check 
only if you notice a difference in load time when 
this option is off—and preferably not on production 
servers. (For more information about the effective- 
ness of disabling the certificate check, see Euan 
Garden’s blog post at _blogs.msdn.com/euanga/ 
archive/2006/07/11/662053.aspx.) 


SSMS can also execute queries in SQLCMD mode. 
SSMS executes these queries by using the Sqlcmd 
utility, letting you include Sqlcmd extended commands 
not available in regular T-SQL. You can toggle this 
mode on and off on the toolbar or from the menu 
by selecting Query, SQLCMD Mode. This setting is 
unique for each query window, and the Query menu 
only shows when the focus in SSMS is on a query 
window. The query window highlights rows containing 
SQLCMD commands, making it easier to remember 
that you're working in this mode. 


Working with Projects 

One of the biggest advantages of SSMS’s use of the 
Visual Studio (VS) paradigm for solutions is that 
it enables you to organize and use your script files 
into solutions and projects. Although SQL Server 
users who have used VS should easily understand the 
concept of projects and solutions, those coming from 
mainly a DBA background can find it a little confusing 
or even not very useful at first sight. 

So what is an SSMS project? It’s a collection of 
connections, script files (with the .sql extension), and 
other files. A project’s main purpose is to organize and 
group related files to make working on a particular task 
or project easier. A solution is a collection of related 
projects, grouped together for ease of management. If 
needed, you can include the same project in multiple 
solutions. You can define one or more database con- 
nections in a project. Each script file is associated with 
the connection it was created with (although you can 
change the connection on the fly if you need to execute 
the script against a different database). 

As a consultant working on multiple projects for 
multiple customers, I’ve come to greatly appreciate the 
ability to group and organize project files as well as 
my collection of saved scripts that I use for almost any 
project I’m working on. It’s also much easier to check 
files into and out of SourceSafe when they’re grouped 
in an SSMS project. Once you understand projects 
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and solutions, you'll see the logic of organizing your 
existing scripts into related projects to make it easier 
to find them and work with them to improve your 
own productivity. 

When you work with projects, one of the frus- 
trating and least intuitive tasks is changing the default 
connection for a project script file to point to another 
database. When you look at the file’s properties, the 
Associated Connection property is grayed out; SSMS 
doesn’t let you change it. Interestingly, if the file has 
no default connection, SSMS lets you type in the 
connection name—and only then “locks” the setting. 
If you change the connection in the query window, 
the file resumes the old connection when you reopen 
the file. So at present, it appears that the only way to 
permanently change the connection associated with 
a script file is to remove the existing connection from 
the project. 

SSMS has many improvements over its predeces- 
sors, but in my everyday work two features stand out 
for me. The first is object filtering: You can now right- 
click almost any list of objects in Object Explorer and 
filter by name, schema, and creation date. I find this 
simple feature to be a huge time-saver when I work 
with databases that have hundreds of tables or stored 
procedures. The other feature I find very valuable is 
the improved dialog boxes. Not only do they open in a 
modal window, letting you switch back to SSMS if you 
need to (or open several dialog boxes at once), but they 
let you capture the generated script instead of executing 
it. This capability can be useful when you want to save 
the script for repeated execution as a scheduled job or a 
stored procedure—as Figure 2 shows for the BACKUP 
DATABASE command—or when you want to examine 
what SSMS executes behind the scenes. 

Unfortunately SSMS still doesn’t provide Intelli- 
Sense to save you from typing the object names in 
full. The buzz is that SSMS in SQL Server 2008 will 
provide IntelliSense for SELECT statements. Until 
then, you can check out SQL Prompt, a third-party 
code-completion tool from Red Gate Software (www 
.red-gate.com). SQL Prompt works as an add-on to 
SSMS, providing IntelliSense and a few other time- 
saving features, such as intelligent SQL code insertion. 
Tve been using SQL Prompt for a while and have found 
that it saves me a lot of time typing in SSMS. 


Working with Reports 

SSMS includes a set of built-in reports to give you a 
better view of what’s going on in the database or on 
the server. These reports are implemented via SSRS 
technology. You can open a report by right-clicking 
an object in Object Explorer and selecting Standard 
Reports. A new feature of SSMS in SQL Server 2005 
SP2 is the ability to include your own custom reports 
and have them easily accessible, like any built-in report. 
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Rolling out your own reports for SSMS is fairly easy 
if youre somewhat familiar with building reports 
using SSRS. Just create a report using SSRS, copy 
the Report Definition Language (RDL) file to any 
folder, and open the report by right-clicking the SQL 
Server instance node in Object Explorer and selecting 
Reports, Custom Reports from the Object Explorer 
pop-up menu. 

After you’ve opened a report, SSMS adds it to 
the list of recently opened reports. SSMS passes in 
a few parameters—such as ObjectName, Object- 
TypeName, Filtered, ServerName, FontName, and 
DatabaseName—from the selected node in Object 
Explorer. Your report can use these parameters, so that 
you can develop context-sensitive reports that are well 
integrated into SSMS. I’ve included a custom report 
called Service Broker Queues with this article, which 
you can download at www.sqlmag.com, InstantDoc 
ID 98764. The UI for Service Broker in SSMS is lim- 
ited, so this report will make it easier for you to see the 
queues in the database, their status, and the number of 
messages in each queue. Note that the report runs in 
the database context and needs to receive the Database- 
Name parameter from SSMS. So, to execute the report, 
you need to select a database node (or any subnode 
below the database node) in Object Explorer. 


Tap into SSMS’s Power 

SSMS in SQL Server 2005 is packed with functionality 

to help you more effectively manage queries, organize 

resources relevant to a management task, and work 

with reports. Explore SSMS for yourself, and let me 

know if you find any more “hidden” features. ER 
InstantDoc ID 98764 
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Scripting a BACKUP 
DATABASE command 
to a new query window 
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DEMING 


the Data Warehouse 


Optimize performance by indexing 
dimensions and fact tables 


few indexes, the data loads quickly but the query 

response is slow. If you have too many indexes, 
the data loads slowly and your storage requirements 
go through the roof but the query response is good. 
Indexing in any database, transactional or warehouse, 
most often reduces the length of time it takes to see 
query results. This is especially true with large tables 
and complex queries that involve table joins. 

Some of the variables that you'll want to take 
into account when indexing the data warehouse are 
the type of data warehouse you have (i.e., primarily 
archive or primarily near real-time), how large the 
dimensions and fact tables are (and if the fact tables are 
partitioned), who will be accessing the data and how 
they'll do so, and whether access will be ad hoc or via 
structured application interfaces. These variables will 
determine how your indexing scheme should be struc- 
tured. Here’s a simple plan for indexing the relational 
tables that comprise a portion of your data warehouse. 
(Although I only explain how to index dimensions and 
fact tables in this article, I explain 
how to index the staging database in 
the Web-exclusive sidebar “Indexing 
the Staging Database,” www.sqlmag 
.com, InstantDoc ID_99494.) Note 
that the relational tables are those 
that are managed by SQL Server's 
relational data engine, not those 
managed by the SQL Server Anal- 
ysis Services (SSAS) engine. 


| ndexing a data warehouse is tricky. If you have too 


Indexing Dimensions 
You'll want to index the dimension 
key (primary key), which is a surro- 
gate key, not a “natural” or transac- 
tional key such as customer name or 
customer ID. Note that you shouldn’t 
cluster on the dimension key. 
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The dimension will contain a natural or transac- 
tional key (e.g., a transaction number or identifier), 
which we'll call a business key, from the source system. 
Although the business key might not be unique—as 
in the case of a type 2 response to slowly changing 
dimensions—create a clustered index on the identity 
column, which you can see in Figure 1. The Customer 
and the Product dimensions have a clustered index 
built on the business key. By clustering on this key, you 
can enhance the query response when the business key 
is used in the WHERE clause. The expression in the 
WHERE clause is often used to search the dimensional 
data, and having the dimension records pre-sorted 
makes the query response faster. 

Clustering by the business key might also help you 
avoid lock escalation (i.e., row to table, intent-exclusive 
to exclusive) during the extraction, transfor- 
mation, and loading (ETL) process, which 
could happen if the surrogate key was the 
cluster key and all the rows were being added 
at the end of the file. If the exclusive locks 
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escalated from row to table, that would block read, 
other ETL load, or utility operations on the table and 
cause diminished query response and even application 
timeouts. If your users and applications can live with 
some latency, you can get around this problem by 
having them query database snapshots and reserving 
the original database exclusively for data loads. 

In Figure 1, the Date dimension and the Time dimen- 
sion have no external datasource or business key. Instead 
of creating an identity-style primary key for these two 
tables, consider using a smart key, witha YYYYMMDD 
format for Date and an HHMMSSSSS format for Time 
(you can use fewer second positions, depending on how 
fine a time granularity you need to measure), and clus- 
tering on it. The values will maintain index order, range 
queries will be simplified in the fact table, and you'll need 
one less join when querying because the primary key will 
contain the date (or time). 

For large type 2 slowly changing dimensions (..e., 
where you add a new row to record the change), you 
might want to create a four-part non-clustered index 
that includes the business key, the record begin date, 
the record end date, and the surrogate key. For effi- 
ciency and to prevent escalating storage requirements, 
INCLUDE the record end date and the surrogate key 
when creating the index instead of making them part 
of the index key, as shown in the following command: 


CREATE NONCLUSTERED INDEX MyDim_CoveringIndex 
ON (NaturalKEY, RecordStartDate) 
INCLUDE ( RecordEndDate, SurrogateKEY) ; 


This command creates a covering index that can be 
useful during the ETL process and load operations 
and for historical queries. When you make RecordEnd- 
Date and SurrogateKEY INCLUDES instead of part 
of the index key, the SQL Server engine stores these 
two values at only the leaf level of the index tree, thus 
reducing the storage requirements. By having these two 
columns in the index (i.e., creating a covering index), 
the SQL Server relational engine can get the data that 
it needs solely from the index during load and some 
query operations, without having to access data from 
the underlying dimension. 

If there are other columns in the dimension that will 
be used continuously for searching, sorting, or grouping, 
create non-clustered indexes on those columns as 
you would in a transactional database. If there’s an 
embedded hierarchy in a dimension, such as the Category- 
SubCategory-ProductID hierarchy in the Product dimen- 
sion, then consider indexing the components of the 
hierarchy if it will enhance query performance and won't 
inhibit data loading. 


Indexing the Fact Table 
Indexing the fact table is similar to indexing a dimen- 
sion, although you must account for partitioning. 


You'll want to index and cluster on the date key or a 
combined date plus time. Because business intelligence 
(BI) analysis always seems to involve a date/time com- 
ponent, the fact table will have a date (or datetime) key, 
and clustering on this key will help with cube-building. 
Also, if the data records are already stored in date or 
datetime order, historical queries will have an execution 
advantage. If the fact table has more than one date or 
datetime column, cluster on the column that’s used 
most often for querying or cube-building. 

If the fact table is partitioned on the date column, 
use that column as the clustering key. When you see 
the same column to create the clustered index that you 
used to create the partitions and creating the index in 
the same file group that holds the partitioned fact table, 
SQL Server will automatically partition the index the 
same way that the fact table is partitioned (i.e., the 
index will have the same partitioning function and 
column as the fact table). When the index is partitioned 
the same way the fact table is partitioned, the table and 
its index are said to be aligned, which makes for an 
optimal operational situation, especially if you antici- 
pate creating additional partitions or making frequent 
partition switches. 

Next, create a non-clustered index on each of the 
foreign keys in the fact table, and consider combining 
the foreign key and the date key, in that order, similar 
to CustomerKEY + DateKEY in Figure 1. Creating a 
non-clustered key on the foreign keys works especially 
well if one or more of the associated dimensions is a 
type 2 slowly changing dimension. Rows with the same 
foreign key value will be searched in ascending date 
order, which will enhance the historical query response. 
Note that you'll want to retain relational integrity when 
dealing with the foreign keys. For more information 
about how to do so, see the Web-exclusive sidebar 
“Retaining Relational Integrity in the Data Ware- 
house” (www.sqimag.com, InstantDoc ID 99362). 


Modifying Your Indexing Scheme 
Over time, your data warehouse will change to accom- 
modate what’s happening in your organization, and 
you'll have to modify your indexing scheme. Most data 
warehouse/BI systems will access these relational tables 
directly, so you can use tried-and-true transactional 
methods for tuning indexes, such as evaluating the 
query and data mix and adjusting it accordingly. If 
your relational data warehouse is used only to stage 
SSAS structures, then you might not need any more 
indexes than those we’ve talked about. SSAS tends 
to use the same queries over and over again, so you 
can run the Index Tuning Wizard and tune exactly for 
that set of queries. Start simple, evaluate thoroughly, 
and build conservatively when indexing your data 
warehouse. SQL 
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In most cases, SQL Server 
deployments are motivated 
by a specific application with 
a database server backend. 
And in a successful business, 
the demands on the database 
continue to grow. As the 
business expands, additional 
database-driven applications 
are deployed and overall 
business growth increases the 
workloads that the SQL Servers 
need to deal with. 


Sizing new data warehouses 
or data marts is different than 
managing existing server 
capacity as workloads grow 
and change over time. This 
report will discuss sizing, 
optimizing, and managing 
your SQL Server infrastructure, 
and you'll find that they are not 
discrete components. Properly 
accomplishing each task will 
allow you to build and run a 
more efficient and more effective 
SQL Server environment. 


Successfully managing the 
growth of your SQL Server 
environment requires an under- 
standing of your users’ query 
volume and mix. In addition, 
Perfmon data gathered over 
time and during peak utilization 
windows will be useful in under- 
standing trends. You also can 
use this same historical perform- 
ance data to effectively optimize 
the SQL environment, reducing 
the amount of time spent in trial 
and error tuning attempts. 


The two most common capacity 
planning techniques involve: 


e “Atomic Modeling” and 
workload profiling, which 
allows management and 
users to play “what-if” games 
with future query mixes and 
demand. In turn, hardware 
utilization may be predicted 
more accurately as workload 
mixes change. 


e Trend analysis 


Trend analysis is useful for 
monitoring resource utilization 
over time. If your future workload 
complexity and volume remains 
“on trend,” your extrapolations 
may be close to meeting your 
workload requirements. 


Assuming that a trend has 

been established, many times 
customers will attempt fo map 
their expected workload to 

a reference configuration. 
Typically, this mapping is 
accomplished by answering a 
series of questions (from a sales 
representative or a “sizing tool”) 
and a vendor would reply with 
a choice from a set of reference 
configurations that they had 
standardized upon. 


While, in some cases, this 
technique may generate a 
configuration that is close to 
what you need, if your users’ 
sophistication and query 
complexity change, or if 
workload volumes deviate from 
the trend, predictions for future 
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hardware capacity will most 
likely have large error factors. 
Therefore, “atomic modeling” 
techniques are a better choice 
because they allow you to play 
“what-if” games by varying the 
workload mix and volume. 


The HP Business Intelligence 
(Bl) Sizer for Microsoft SQL 
Server allows you to profile your 
specific workload mix (query 
complexity) and couple that 
with a query volume (query 
per hour rate). By using similar 
“atomic modeling” algorithms 
used by sophisticated capacity 
planning tools, the HP sizing 
tool for Microsoft SQL Server 
BI will generate a configuration 
to support the workload 
defined to the tool. This is very 
different from a “sizing tool” 
that is simply a “reference 
configuration selector.” 


As you can see, the key to 
properly sizing and optimizing 
your environment is to have a 
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strong management solution in 
place. Such a solution would 
allow you to use historical 
management data to generate 
reports about the behavior and 
performance of the SQL Servers 
and the hardware they run on. 


This information needs to 

track the behavior of the user 
environment, the workloads 
generated, and the impact 

of the workload on the 
performance and utilization 

of the SQL Servers and I/O 
subsystem themselves. The more 
information you have at hand 
about workloads and system 
behavior, the better the choices 
you can make when sizing new 
hardware or adding additional 
hardware to the environment. 


While trend analysis is generally 
a straightforward process, it can 
be time-consuming depending 
upon the number of servers 
being monitored. Trend analysis 
is also easy for most people to 
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understand because viewing 
utilization graphs over time is 
frequently intuitive. 


The wrinkle comes if your 
workload mix changes 

(i.e., users become more 
sophisticated and request more 
complex queries or query 
volumes may increase due 

to company or department 
mergers). Trend analysis, 

by its nature, assumes that 

the workload mix of simple, 
medium, and complex queries 
remains consistent with what 
it was in the past. Because 
“atomic modeling” is more 
flexible than trend analysis 

it allows people to predict 
resource utilization more 
accurately as workloads vary. 


For the purposes of this report, 
we consider sizing to be the 
process of determining the 
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proper server hardware and 
storage infrastructure for your 
SQL Server environment. 
Accurate sizing information 
gives you the best chance of 
deploying a configuration that 
can be smoothly integrated into 
an existing environment or used 
to upgrade and replace an 
existing hardware infrastructure. 
This concept includes the 
traditional thoughts about sizing 
a database, but our focus is on 
selecting the proper server and 
storage hardware mix that will 
deliver the optimum platform for 
your SQL Server enterprise. 


Improperly sized hardware 
usually falls into two common 
categories: too large or too 
small. If the choice is simply to 
invest in as much of the biggest 
and fastest hardware that the 
budget can afford, in most cases 
you simply will be wasting IT 
budgetary resources that can be 
better spent elsewhere. 


Excess capacity —that is, 
capacity beyond what is 
needed for optimal database 
performance and expected 
near term growth — usually 
implies that the budget has not 
been spent effectively. As the 
underlying technologies change 
and improve, any previous 
unnecessary investment limits 


budgetary flexibility in the future. 


If the supporting hardware 
infrastructure is insufficient to 
support the demand being 
placed on the SQL Server 
databases, the result is usually 
poor application performance. 
Undersized servers or storage 
devices are a common cause 
of performance problems, an 
issue that can be verified by 
looking at the management 
records for the servers. If you're 
seeing sustained CPU utilization 
of more than 85 percent, it’s 

a good indication that the 
workload on your servers is 
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too extreme; other features 
of a good management tool 
can indicate performance 
bottlenecks. 


In data warehousing 
applications storage is a 

very common performance 
bottleneck. A good sizing tool 
can address a proper storage 
configuration, ensuring that a 
sufficient number of disks and 
controllers are configured to 
balance performance with the 
number of processors in the 
server. 


The bottom line is that a 
properly sized SQL Server 
infrastructure will be able to 
deliver the performance and 
capabilities required for the 
business processes supported 
on the server. An improperly 
sized server will be a source of 
problems and disappointments 
for IT and the businesses it 
supports. 


SUSTAINED CPU UTILIZATION 
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To provide accurate data for 
your SQL Server sizing needs, 
you need to understand your 
database environment. Common 
questions that need to be 
answered include: 


e What types of applications 
are you running? 


e How many users are 
accessing the data? 


e What types of users 
are accessing the data 
(knowledge workers, power 
users, developers)? 


e What types of queries are 
being generated? 


e How many queries are being 
generated per minute/hour/ 
day? 


e How many rows must be 
loaded into your data 
warehouse? During what time 
window? 


The most common business 
applications revolve around 
transaction processing and BI. 
These classes of applications 
have their own requirements 


for the configuration and, 
therefore, the sizing of the 
underlying server and storage 
infrastructure. Applications 
such as those focused on BI 
are often multi-tiered, with their 
own requirements for each 
tier of the supporting structure. 
Common BI applications 
require configuration for such 
functionality as 


e On-line Transaction Processing 


(OLTP) 


e On-line Analytical Processing 
(OLAP, sometimes referred to 
as MOLAP) 


Relational On-line analytical 


Processing (ROLAP) 


e Extract, Transformation, and 


Load (ETL) 


User numbers are also a critical 
part of the sizing picture. But 
asking how many users access 
your databases will often not 
generate an accurate answer. 
This is because it is not solely 
the total number of users you 
have that affects the workload 
on the SQL Servers, but rather 
what types of queries those 
users are running and how 
often. In addition, the term 
“simultaneous users” may cause 
contusion (e.g., Are 
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these users who are allowed to 
use the system? Users who are 
logged on simultaneously? Users 
who actually hit “enter” simul- 
taneously?) Therefore, quoting 
system performance in terms of 
“Queries per Hour (QPH)” tends 
to make the most sense. 


In general, the vast majority 

of users will be running 
standardized queries or canned 
reports. In these situations, 
where many users are accessing 
the same standardized reports, 
it’s likely that fewer queries 

are being run than if users are 
querying for unique data. 


More advanced users are those 
who are likely to be working with 
the data that the average user is 
just generating reports against. 
They may well be using the same 
data set and manipulating that 
data, so they put a higher load 
on the database; yet they still are 
not doing anything that would 
seriously tax performance since, 
in most cases, they are working 
with previously aggregated data. 


These first two groups of 
users impact the database by 
their sheer numbers; in some 
organizations there may be 
hundreds if not thousands of 


users who interact daily with 
the SQL Server applications in 
this fashion. Fortunately, their 
impact is relatively quantifiable. 
If you're developing new 

BI environments, you will 

have to interview users to 
determine how many users 
will be accessing the system 
at the same time. You should 
also take into consideration 
average think time between 


queries and query response 
time, which will allow a vendor 
to estimate the QPH workload 
on the server. (Note: The HP 
sizing tool for Microsoft SQL 
Server BI guides you through 
this process and calculates the 
QPH rate for the analysis.) On 
the other hand, if you have an 
existing Bl environment you 
may use management tools 
that monitor the traffic, queries, 


and performance behaviors of 
SQL Server and the underlying 
hardware, to give you a very 
accurate idea of what your 
ongoing workload looks like. 


The most difficult user impact 
to evaluate is that of the power 
user. These are the users and 
developers who often generate 
complex ad-hoc queries that 
potentially scan tables across 
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A HP Business Intelligence Sizer for Microsoft SQL 
Server 2005 


Figure 1: 
The HP BI Sizer for 


main screen 


Microsoft SQL Server 2005 


HP BI 
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an entire database. Because 
their ad hoc queries can cause 
a significant hit on the resources 
available to the database 
server, the best way to plan 

for their usage is to use your 
management tools to look at the 
overall end-user usage heuristics 
and attempt to determine what 
percentage of that use can be 
attributed to your power users. 


The same management and 
monitoring solution should be 
able to produce a queries- 
over-time report that tells 

you the number of queries 
being generated. With this 
information, you can determine 
when workloads are at their 
highest and what the current 
peak query/transaction rate is. 


Different configurations of 
server and storage are required 
based on the type of processing 
occurring. OLTP, OLAP, ROLAP, 
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and ETL all place a different set 
of demands on the servers and 
storage hardware and the way 
that you decide to design and 
configure your databases and 
policies. The right sizing tools 
will be aware of the different 
requirements for the various 
processing and data loading 
options. 


Sizing tools come in a variety of 
shapes and sizes, ranging from 
expensive fools dedicated to 
helping you properly size your 
databases for specific software 
to a broad selection of free tools 
that help you select the proper 
hardware configurations for your 
database servers and storage. 


There are major differences 
between the sizing tools 
available from various 
hardware vendors. In most 
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cases, the tools aren't actual 
sizing tools; rather, they are a 
reference configuration selector. 
These tools ask a series of 
questions and then point you 
to a selection from a list of 
reference configurations. One 
set of tools that offers much 
more realistic and effective 
sizing recommendations comes 
from HP. The HP BI sizer and 
the OLTP sizer for Microsoft 
SQL Server use real-world 
benchmarked workloads 

to provide a reasonable 
workload configuration and 
actual calculations against that 
workload. 


The HP BI sizing tool for SQL 
Server (Figure 1) converts the 
number of users into a Query 
Per Hour rate and models a 
mixed workload of simple, 
medium, complex, and very 
complex queries. The tool gives 


Figure 2: 


HP Business Intelligence Sizer for Microsoft SQL Server 2003) 


Informational 
Message from the HP 
BI Sizer for Microsoft 
SQL Server showing 
the metric used in 
defining a “Normal” 
OLAP user. | 


Normal OLAP Users 


Normal users typically access a small or medium number of cells and do not perform 
complex calculations or data conversions with the data. | 


you the option to change the 
number of I/Os per query 
type, in case you believe that 
the assumed I/O rate is lower 
or higher than the industry 
standard benchmarks used to 
create the model for the tool. 


Once the workload is defined 
and related to the database 
size, the sizer models and 
calculates an estimated 
megabyte per second |/O 

rate and the amount of CPU 
resources required to support 
the pre-defined workload. 
These metrics are then mapped 
to physical hardware (HP 
Integrity or HP ProLiant servers) 
to determine the appropriate 
amount of processing required 
for the pre-defined workload. 


In a similar fashion, MB/ 
second rates are also used to 
calculate the number of disk 
drives and controllers required 
to support your throughput, 


while maintaining the defined 
response times entered into the 
tool. Based on the data you 
enter into the sizer tool, the 
tool uses “atomic modeling” 
algorithms to generate the 
combination of HP server 
hardware and HP storage 
solution that will best meet your 
needs. (The HP BI Sizer for 
Microsoft SQL Server is not a 
reference configuration selector.) 


The HP BI Sizer for SQL Server 
can recommend configurations 
for any of three SQL Server 
roles (or any combination of the 
three): 


e Load Processing — SQL Server 
Integrations Services (ETL) 


e Database Query Processing — 
SQL Server Data Warehouse 
(ROLAP) 


e Cube Query Processing - 
Queries requesting data from 
SQL Server Analysis Services 
(OLAP) 


The HP BI Sizer for SQL Server is 
aware of the differences and the 
requirements for each of the roles 
defined. You are offered context- 
aware choices that change as 
you make your selections. Only 
the options appropriate for your 
selected choices appear as you 
work your way through the tool. 
For entries that require you to 
make choices based on what 
type of workload to expect, 
linked information boxes tell you 
what metric was used to define 
the entry, or the type of user the 
entry represents, as shown in 
Figure 2. 


After you walk through all the 
HP BI Sizer for SQL Server 
information screens and enter 
the requested data, the tool 
takes the data entered, applies 
its formulas to the calculations, 
and offers up a solution—along 
with information about the 
selection choices. You can drill 
down for detailed information 
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about the recommended 
custom configuration, as well 
as alter the individual pieces of 
the solution to add or modify 
options and accessories. 


Figure 3 shows one of the 
customization screens for a 
solution generated by the sizer 
tool. Clicking on the highlighted 
links on the screen, in this case 
the word “server,” opens a 
browser window to the HP Web 


site, which provides information 
about the product recommended 
in the dropdown box. 


With an Internet connection 
the HP BI sizer automatically 
updates itself and the pricing 
information it uses. This means 
that the data it provides is 

as current as the most recent 
update. Pricing information 

is updated whenever prices 
change. You can perform 
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different types of “what if?” 
simulations within the sizer, 
saving all of the result sets and 
modifications and exporting 
the results to Microsoft Excel 
or Word format files. Links are 
provided to Web content on 
HP reference configurations 
for BI solutions, and if you 
need it, there is a direct email 
link to HP for assistance in 
configuring custom enterprise 
BI solutions. 


Figure 3: 
Solution 
Customization Screen 
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Figure 4: 
The HP OLTP Sizer 


The HP OLP sizer (Figure 4), 
designed similarly to the HP BI 
sizer, also uses real experiential 
information derived from 
industry-standard benchmarks 
for modeling a database load 
and to establish hardware 
recommendations based on your 
OLTP SQL Server environment. 
The HP OLP sizer handles sys- 
tem recommendations, custom- 
izes your solutions, and exports 
the saved configurations in the 
same way as the HP BI sizer. 
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The number one rule of thumb 
for properly sizing hardware for 
your SQL Server infrastructure 

is “know your environment.” 
Understanding how your data- 
bases are used, what types of 
transactions are being run, how 
often, how complex, and by how 
many users goes a long way 
toward giving you the knowledge 
and tools to properly size your 
SQL Server hardware for opti- 
mum performance and efficiency. 


SQL Server hardware 
optimization requires that you, 
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ProLiant Transaction Processing Solutions for 
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once again, understand exactly 
what is going on in your SQL 
Server environment. The types 
of applications being run, how 
they are being used, and the 
behavior of those applications 
on the underlying hardware all 
affect how the hardware should 
be optimized to deliver the most 
effective performance. 


Server—The various server 
subsystems each benefit from 
closer examination. Generally, 
the concept of more is better 
applies, but the point of 
diminishing returns can be 
reached quickly in some 
situations. 


CPU—With the current 
generation of CPU technology, 
you can take good advantage 
of the per processor licensing 
model for Microsoft Windows 
Server and Microsoft SQL 
Server, especially in a high- 
performance, heavily used 
environment with a high client- 
to-server ratio. This is because 
the licensing model treats a 
single core CPU the same as 

a multi-core, so the processing 
advantages of dual- and quad- 
core CPUs inflict no additional 
licensing charges. 


So how many CPUs are 
appropriate? If you have 
historical data on the behavior 
of the applications, you want to 
be sure that your CPU utilization 
levels are such that the CPUs 
provide adequate response 
time —even at times of highest 
use. If you've used a sizing tool, 


Broadcom 5704 Gigabit Ethernet 
Embedded 10/100/1000T WOL 


such as described earlier in this 
report, you'll see that you have 
the ability to define peak CPU 
utilization. Future plans should 
be considered when specifying 
CPUs (type and number) or 
sizing for peak utilization. If 
you plan to add additional 
applications or expand the 
business, specifying additional 
CPUs or changing from dual- 
core to quad-core processors 
makes sense. When using sizing 
tools, a good rule of thumb is 
that CPU utilization should be 
specified at below 85 percent; 
if you plan for expansion, lower 
the peak utilization you are 
willing to accept. This will cause 
the HP sizing tool to model for 
additional CPU capacity. 


Memory—With 64-bit 
Windows Server and SQL 
Server the amount of memory 
that you can install in the 
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server is currently limited by 
memory density and server 
motherboards. The traditional 
rule of thumb for SQL Server 
memory size is to have enough 
memory to put the largest 
database entirely in memory. 
This remains an excellent idea, 
but as database size grows 
into the hundreds of gigabytes, 
terabyte, or petabyte range, 

it is rarely practical due to the 
current memory limitations of 
Windows Server hardware. 


The type of use planned can have 
a direct impact on the amount of 
memory installed. For example, if 
the target is a huge database that 
is not practical to fit into memory, 
but the primary use is OLAP using 
a data cube, the ability to fit as 
much of that cube into memory 
as possible will improve the user 
response time for queries made 
against the cube. 


Port Type 


10/100/1000-T 
Gigabit Ethernet 


Integrated 


10/100/1000-T 
Gigabit Ethernet 


Figure 5: 
Sample Server 
NIC configuration 
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Management 


Integrated 
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To a certain degree, the tradeoff 
becomes a price/performance 
challenge. High-density memory 
is expensive, and adding 
hundreds of gigabytes of 
memory to your SQL Server, 
while possible in some cases, is 
expensive. You must be able to 
weigh the initial purchase price 
against the ongoing benefits of 
reduced response time for end- 
user data requests. 


1/O—Disk I/O will be 

a function of the storage 
model being chosen and the 
intended role of the server. 
High-performance local disk 
controllers (usually serial- 
attached SCSI controllers and 
drives) for the server operating 
system local storage will likely 
be supplemented by fiber- 
channel attached controllers. 


Network —Gigabit Ethernet is 
the standard in server hardware 
and enterprise-class server 
hardware will often have multiple 
integrated GbE controllers, as 
shown in Figure 5. 


The decision to use network- 
based storage technologies, 
such as iSCSI, will impact 

the NIC configuration in your 
servers. For example, the choice 
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of implementing iSCSI in a data 
center may mandate the use of 
10 GbE, rather than 1 GbE, 

in order to meet performance 
requirements. 


Storage 


Storage optimizations depend 
largely upon the type of 
applications that are accessing 
the storage. There are general 
considerations, which will 
enable SQL Server to deliver its 
best possible performance. 


e More spindles almost always 


equates to better performance. 


For delivering low latency 
response times to end users, 
the size of the drives is less 
important than the number of 
spindles and the speed of the 
drives. 


Don’t overanalyze the storage 
configuration; simpler is 
usually better. Design for 
growth potential, not for the 
immediate need. It’s always 
easier to expand storage 
systems that were designed 
with the future expansion of 
applications, users, and data 
in mind. 


e Understand how SQL Server 
I/O works and how your 
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applications affect the use of 
I/O. An OLTP environment 

is far different from a ROLAP 
implementation. For example, 
typical OLTP I/Os are 8KB or 
less. But in data warehousing, 
it is common for table scans 
to perform 300KB to 400KB+ 
per I/O request. These vastly 
different |/O profiles affect 
storage design issues. Bear 
in mind that the HP sizing 
tools are aware of these 
differences and propose 
storage subsystems to support 
each workload’s respective 
characteristics. 


Consider RAID 10 for the 
location of log files and 
TEMPDB. This provides 
hardware protection for 
failures and fault tolerance for 
the log files and an increase 
in performance if TEMPDB is 
heavily used. 


SAN — Storage Area Networks 
offer the greatest flexibility for 
a SQL Server infrastructure. 
With a huge number of 
configuration options, flexible 
storage architectures, multiple 
connection topologies, and 

the ability to offer techniques 
such as storage virtualization, 
the choice of a SAN as the 


storage architecture is usually 
an enterprise-driven choice, 
rather than an application 
specific one. If the operational 
environment allows the use of 

a SAN architecture, you can 
achieve significant benefits in 
management, virtualization, and 


ROI by its use. 


However, SAN configurations in 
general enterprises are usually 
focused on techniques such 

as file space optimization. But 
OLAP and BI workloads require 
that the SAN be optimized for 
I/O performance. In addition, 
as noted earlier, care should be 
taken to optimize for 400KB-plus 
requests that may simultaneously 
compete for 8KB requests 
generated by OLTP applications. 
In many cases, zoning or having 
a dedicated SAN may be useful 
for optimizing workloads with 
very dissimilar characteristics. 
Microsoft offers a detailed 
article on I/O performance 
requirements for SQL Server 

at htto://www.microsoft.com/ 
technet/prodtechnol/sql/ 
bestpractice/pdpliobp.mspx 


DAS —Direct Attach Storage 
can offer the highest 
performance for your SQL 
Server databases. High- 


performance array controllers 
in the server hardware can 
deliver the best possible 

I/O performance from 

DAS devices attached to 

the server. In environments 
where the maximum possible 
I/O throughput is the key to 
successful deployment, such as 
enterprise OLTP installations, 
DAS might be the optimal 
choice for the infrastructure. 


RAID—As you'll find 
mentioned in numerous white 
papers posted on Microsoft's 
Web site, SQL Server tends 
to be deployed in RAID 
configurations, with RAID O, 
1, 5, and 10 being the most 
common choices. 


RAID O stripes data across all 
the disks in a set and offers the 
highest performance at the cost 
of no fault tolerance. The failure 
of any disk in the set will result 
in lost data. 


RAID 1 is disk mirroring. In this 
setup the contents of each disk 
are mirrored exactly, providing a 
redundant copy of the disk data. 
This configuration offers fault tol- 
erance and may improve read 
speeds, but at the cost of a slight 
degradation in write performance. 
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RAID 5 is striping with parity. 
It has all the data and parity 
information striped across all 
the disks in the set. It provides 
fault tolerance and can survive 
disk failure, although with 
reduced performance until the 
missing disk is replaced and 
rebuilt. 


RAID 10, also known as RAID 
1+0 is disk striping and disk 
mirroring. The entire stripe set is 
mirrored to another set of disks. 
This provides the performance 
benefits of striping with the 
fault tolerance of mirroring 

and offers the highest read/ 
write performance of any of 
the standard RAID levels. The 
tradeoff here is the requirement 
for 100 percent redundancy in 
the number of disk drives. 


Other levels of RAID exist 

and may offer additional 
enhancements for performance 
and fault tolerance. Usually, 
these RAID levels are proprietary 
to specific hardware vendors. 


Data protection goes beyond 
simply providing a fault-tolerant 
storage system for the SQL 


Figure 6: 

The HP OpenView 
Monitoring 
Architecture 

for SQL Server 


Server infrastructure. A number 
of solutions can deal with the 
backup and storage of the 
database information, ranging 
from tape drives to continuous 
data replication solutions. 


At the cutting edge of 
technology at this time are 
solutions that support disk-to- 
disk-to-tape backup, providing 
high-performance disk-to-disk 
backup of running databases 
with the ability to copy the 
disk backup to tape, allowing 
for off-site storage of backup 
tapes. The data protection 
scheme you choose should 
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be part of the comprehensive 
information lifecycle 
management plan in place 
within your organization. 


With optimization, the goal 

is to match your SQL Server 
environment to the needs of your 
applications and users. While 
sizing and optimization tools 
can give you a good starting 
point for your environment, a 
detailed understanding of the 
behavior of your applications 
and users is an absolute 
requirement for a successful 
deployment. 


Ensuring optimum performance 
is challenging: You have too 
many things to watch and 
monitor with a single SQL Server, 
let alone with a multi-server 
environment. No successful 

SQL Server deployment and 
operation can be done without 
a good centralized management 
application. A good 
management tool should deliver 
an end-to-end management 
experience by monitoring, 
managing, and reporting on 

the behavior of the SQL Server 
databases, the underlying 


ptimizing, and Managing your 


operating system software, and 
the hardware infrastructure. 


HP OpenView offers a compre- 
hensive management solution 
that is capable of managing 
the SQL Servers deployed in 
an enterprise, the Windows 
Server operating system on 
which SQL Server resides, and 
the hardware that Windows 
Server is running on. This is 
accomplished with a series of 
specialized plug-in modules for 
the OpenView console, such as 
the Smart Plug-in for Microsoft 
SQL Server, which adds 
application-specific capabilities 
to the OpenView Platform 

(as shown in the architecture 
diagram in Figure 6). 


Effective management requires 
this overall approach due to the 
interaction of SQL Server with 
the underlying operating systems 
and hardware. 


Training and consulting services 
Developing, deploying, and 
managing a mission-critical 
SQL Server infrastructure isn’t 
easy, and with the significant 
size of the investment that needs 
to be made in hardware and 
software, investing in services 


that can get you up and running 
more quickly and effectively is a 
logical choice. 


For example, properly 
configuring a SAN for use 

with SQL Server can be very 
complex; not simply the selection 
of hardware and drives, defining 
the number of spindles, laying 
out the various databases, temp 
files locations and database 
sizing, but selecting the switches, 
fabric, and topology that can 
deliver the performance required 
by the current and potential 

SQL Server applications. Very 
few organizations have this 

kind of expertise in-house 

and using it upfront, prior to 

the implementation, can be a 
significant cost saver both in IT 
time and in long-term operational 
costs. 


Training can jump-start your IT 
team in the use of management 
tools and applying those tools 
to the needs of the particular 
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situation IT faces on a regular 
basis. Integrating new tools 
with those currently in use or 
upgrading existing tools to add 
new capabilities can require 

a learning curve that can be 
greatly accelerated with the 
proper training. 


Successfully investing in a high- 
performance SQL Server solution 
that delivers what the end 

users need and expect requires 
a detailed understanding of 

the needs of SQL Server, the 
applications that are planned 

to be deployed, and the needs 
of the end users. With the 
software used to manage and 
monitor existing SQL Server 
applications, you can define the 
information and data that you 
need to properly size, optimize, 
and continually manage your 
investment in Microsoft SQL 
Server and its underlying 
hardware and software. 


is a Senior Contributing Editor for Windows 
IT Pro magazine. He has been writing computer-related features 
and product reviews for more than 20 years, has been the CTO 
of a network management tools ISV, and is currently running a 
small consultancy for businesses in the SMB space. 


SQL Server Profiler or 
Server-Side Trace? 


That is the question 


for the performance-conscious DBA 


*m sure you're familiar with the built-in SQL Server 

Profiler tool and the wonderful information it can 
return and display for you. Most DBAs use the tool 
on a weekly—if not daily—basis to troubleshoot 
problems or to simply look for poorly performing 
queries. I’m not here to explain what the tool does but 
rather to answer an increasingly popular question that 
people seem to have about the tool. You see, there’s 
an ongoing debate—raging since the debut of SQL 
Server 7.0—about what kind of performance impact 
SQL Server Profiler has on overall performance of the 
database server when it’s actively tracing. 

Whenever you click Run while using the tool against 
a production server, you probably wonder about the 
performance hit. I know there are also DBAs and 
developers among you who refuse to believe that SQL 
Server Profiler might adversely affect the server and 
therefore use it in all situations. Make no mistake: SQL 
Server Profiler can impose a performance penalty, and 
in some cases that penalty can be severe. 


Profiler vs. Trace 

The alternative to SQL Server Profiler is—and has 
always been—performing a server-side trace. Infor- 
mation about the trace procedure is readily available 
(see “9 Steps to an Automated Trace,” InstantDoc ID 
43014, for information about how to create and use 
a server-side trace), but suffice it to say, server-side 
traces bypass the SQL Server Profiler GUI and send 
the trace-event information directly to a file on the 
local hard disk. Then, you can later read the trace file 
for further processing or viewing, through SQL Server 
Profiler or the fn_trace_gettable function. Common 
wisdom is that server-side traces require significantly 
less overhead than SQL Server Profiler to process the 
same number of events. To see just how much less, I 
decided to put the two procedures to the test. 

Keep in mind that many factors ultimately deter- 
mine the amount of stress each of the two options will 
put on the server. For SQL Server Profiler, these factors 
include whether you run the tool locally or on a remote 
client; the network’s speed and overall efficiency; the 
client’s processors, available memory, and disk speed; 
and SQL Server Profiler options (e.g., normal trace, 
trace-to-file, trace-to-table). 
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For a server-side trace to a file, the only applicable 
option is where the trace file will be located. Tracing to 
a file on a network share isn’t recommended, but it’s an 
option. Tracing to a file located on the same disk array 
as the data or log file also isn’t recommended, because 
of the extra overhead it might incur on the trace. The 
preferred method is to trace to a file on a local drive 
(either SAN or direct-attached) that doesn’t have any 
other traffic that will impede the writing or interfere 
with other processes accessing that disk. 

For the purposes of this test, I tried to keep everything 
as simple as possible by limiting the number of events and 
columns to that which would be typical of most traces. 
Therefore, I’m capturing only the Batch Completed and 
RPC Completed events, with the default columns. Keep 
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user groups. 


in mind that, in both cases, psen pR Oo 
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the more events and col- 
umns you choose to trace, 
the larger the impact you'll 
have on the server while 
tracing. To simulate a load 
on the SQL Server system, 
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performance impact at 
reasonable load 


Figure 2 


Server-side trace 
performance impact at 
reasonable load 
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i i PROFILER OR TRACE? 


and two that used server- the trace and the blue line indicates the end. Figures 1, 
side traces. Then, I ran page 47, and 3 show the results of running SQL Server 
two sets of tests for each Profiler, and Figures 2, page 47, and 4 show the results 
group, using both a single- of using server-side traces. As you can clearly see, SQL 
user load and a two-user Server Profiler has a significant performance impact and 
load. The key was to show server-side traces have an almost negligible effect. You 
the effect on the server can also see that the impact gets much more pronounced 


ny when processor utiliza- when the number of batch requests per second and server 
RI , i | tion is both midrangeand CPU usage increase. In the first set of tests, the server 
oe OON SONETO NS nearly maxed out. was averaging about 4,000 batch requests per second 
nD OR rens teen e FE Inall four of thisarticle’s without SQL Server Profiler running and about 3,000 
thon Color Sewe Cotes bertarce Penre Obes figures, the vertical green with it running—a 25 percent decrease in overall system 
2 San tate nga line indicates the start of throughput. In the second set of tests, the results go from 
Figure 3 just over 6,000 requests per second to about 2,000 requests 
Be n-i¢xF .o048) +n a per second. That’s a 66 percent decrease in throughput 
scat a jen a A” a E while using SQL Server Profiler, whereas the impact of 
heavy load | the server-side trace remained hardly noticeable. 

Choose Wisely 
My intention isn’t to convince you to stop using SQL 
Server Profiler altogether on a live server, but rather to 
give you some food for thought. Adding a proper filter 
: to a server-side trace to limit the captured events can 
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Sharpen Your Basic 
SQL Server Skills 


Learn how to join a table to itself using a self-join 


Q: What is a self-join? 

A: A self-join is a standard SQL Server join that joins a 
table to itself. You can perform a self-join when all the 
data you require is in a single table, and the data you need 
to extract relates to other data in the same table. You join 
a table to itself using a field or combination of fields that 
have matching data in different records. To perform a 
self-join, you must use a table name for each instance of 
the table, so that each instance has a separate name. Use 
a self-join when you want to compare values in a column 
to other values in the same column. The data type of the 
columns on which the join is made must be the same or 
you'll need to cast them to the same type. 

For example, in employee information records, each 
record might have a field for an employee’s ID number 
and also a field for the supervisor’s ID number. To 
retrieve a list of employees and their managers you 
need to join the employee information table to itself: 


Q: Is a self-join an inner join or an outer join? 
A: A self-join can be either an inner join or an outer join. 
Let’s look at some examples that show how using either 
an inner join or an outer join affects the self-join. 

First, run the following code to create a table called 
SaleOnDate with two columns, DateOfMonth and 
Sold-Qty, and populate it with sample data: 


USE TestData 
GO 
CREATE TABLE SaleOnDate (DateOfMonth 
INT, SoldQty INT) 
GO 
INSERT INTO SaleOnDate (DateOfMonth, 
SoldQty) 
SELECT 1, 
UNION ALL 
SELECT 2, 
UNION ALL 
SELECT 3, 
UNION ALL 
SELECT 4, 
UNION ALL 
SELECT 6, 
UNION ALL 
SELECT 7, 288 


108 
158 
208 
225 


258 
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UNION ALL 
SELECT 8, 308 
GO 


The SaleOnDate table contains information about 
the day of the month and the quantity of items sold 
on that day. Figure 1 shows the contents of this table, 
which you can see by running the following query: 


SELECT * FROM SaleOnDate 


Next, run the following query to find the difference 
between a named day and the next day: 


SELECT sd1.DateOfMonth, (sd2.SoldQty - 
sd1.SoldQty) DifferenceInSale 
FROM SaleOnDate sd1 
INNER JOIN SaleOnDate sd2 ON sd1. 
DateOfMonth + 1= sd2.DateOfMonth 
GO 


This query demonstrates the use of an inner join when 
joining the SaleOnDate table to itself. As you can see 
in Figure 2, the result set doesn’t contain information 
about days 4 and 8, because days 5 and 9 don’t exist in 
the database. This is because the conditions specified 
in the join clause haven’t been met. 

Next, let’s look at the use of a left outer join, also 
called a left join, to join the SaleOnDate table to itself. 
The following code demonstrates the use of a left join 
when joining the SaleOnDate table to itself: 


SELECT sd1.DateOfMonth, 
ISNULL CCAST((sd2.SoldQty - sd1. 
SoldQty) AS VARCHAR(1@)), ‘No Data’) 
DifferenceInSale 
FROM SaleOnDate sd1 
LEFT JOIN SaleOnDate sd2 ON sdl. 
DateOfMonth + 1= sd2.DateOfMonth 
GO 


The result set, shown in Figure 3, now contains informa- 
tion about days 4 and 8. Because days 5 and 9 don’t exist in 
the database, the result set contains NULL for days 4 and 
8. NULL values are displayed as No Data because the 
ISNULL function was used in the SELECT clause. EA 
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E Results |} Messages! 
DateDfMonth SoldOty 
1 100 
2 2 150 
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7 8 300 
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T=S@L 101, 
Lesson ð 


until now, the examples in the 

T-SQL 101 series have accessed 

tables that already existed or were 
created by running T-SQL code included with the 
lesson. Now TIl show you two techniques you can use 
to create your own tables. The first technique uses the 
CREATE TABLE command. The second technique 
uses a SELECT INTO command. 

With the CREATE TABLE command, you specify 
the new table name and define the table’s column struc- 
ture. With the SELECT INTO command, you use a 
SELECT statement to query a database and store the 
results in a new table, as specified by the INTO clause. 
The column definitions of the new table exactly match 
the column definitions in the result set. 

At this point, you might be wondering, “Isn’t it 
a little archaic to manually create tables with T-SQL 
statements when there are graphical tools you can use 
to create tables?” My response is that no matter what 
tool you use to design tables, the tables don’t actually 
get created in the database without T-SQL statements 
being generated, submitted, and executed. Much like 
an artisan using a hand planer rather than an electric 
sander to smooth wooden surfaces, I find that cre- 
ating tables “the old-fashioned way” leads to a better 
understanding of the tables and the columns therein. 
Manually typing the T-SQL code forces you to slow 
down and seriously consider the names and data types 
you assign to each column, and how they relate to other 
tables in the database. A T-SQL table creation script 
also documents the table’s structure and lets you easily 
recreate the table. 


The Prerequisites 

To run the examples in this lesson, make sure your data- 
base contains the Employee table created in Lesson 3 
and the Movie and Genre tables created in Lesson 5. 
If you haven't created these tables, you'll find the code 
in the 99448 zip file, which you can download by going 
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How to create tables 


to www.sqimag.com, entering 99448 in the InstantDoc 
ID box, and clicking the 99448.zip hotlink. 


Using the CREATE TABLE 
Command 
CREATE TABLE is one of those syntax-heavy com- 
mands that can be a bit intimidating at first. Instead 
of rehashing the excellent documentation for this 
command in SQL Server 2005 Books Online (BOL) 
at msdn2.microsoft.com/en-us/library/ms174979.aspx, 
T’ll just cover the basics to get you started creating your 
own tables. After you become familiar with the basics, 
I highly encourage you to delve into this command’s 
complexities. 

The basic syntax of the CREATE TABLE com- 
mand is 


CREATE TABLE DatabaseName. 
SchemaName. Tab1eName 

¢ 
ColumnNamel datatype, 
ColumnName2 datatype, 
ColumnName3 datatype 

) 


DatabaseName.SchemaName.TableName is referred 

to as the fully qualified table name, where 

* DatabaseName is the name of the database in which 
the new table will reside. This argument is optional. 
Tf left out, the table will be created in the current 
database. 

e SchemaName is the 
schema to which the 
table will belong. A 
schema is a collection 
of uniquely named 
objects that aren’t 
automatically tied to 
a particular user. This 
argument is optional. If 
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LISTING |: Code to Create the New 


MovieReview Table 
USE MyDB 


-- Drop the old table if it exists. 
IF OBJECT_IDC'MovieReview') IS NOT NULL 


DROP TABLE MovieReview 
ee) 
CREATE TABLE MovieReview 
¢ 


MovieReviewID int IDENTITY (1,1) PRIMARY KEY, 
EmployeeID int NOT NULL, 


MovieID int NOT NULL, 
Stars tinyint NOT NULL 
) 


LISTING 2: Code to List Who Reviewed 
Which Movies, the Reviewers’ Ratings, and the 


Movies’ Genres 


SELECT m.Movie, 
g.Genre, 


LEFT(e.FirstName + ' ' + e.LastName,2@) AS 'Reviewer', 
LEFTCREPLICATE('* ',mr.Stars),10) AS 'Stars' 


FROM MovieReview mr 


INNER JOIN Employee e ON e.EmployeeID = mr.EmployeeID 
INNER JOIN Movie m ON m.MovieID = mr.MovieID 
INNER JOIN Genre g ON g.GenreID = m.GenreID 


WHERE mr.Stars > 3 


ORDER BY mr.Stars DESC, g.Genre, m.Movie 


LISTING 3: Code to Create and Populate 
the MovieReviewSummary Table 


SELECT m.Movie, 
g.Genre, 


LEFT(REPLICATE('* ',AVG(mr.Stars)),1@) AS 'Stars', 


COUNT(*) as 'Reviews' 
INTO MovieReviewSummary 
FROM MovieReview mr 


INNER JOIN Movie m ON m.MovieID 
INNER JOIN Genre g ON g.GenreID 


GROUP BY m.Movie, g.Genre 
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left out, the table will be owned by the dbo schema, 
which is the default schema of the database owner 
(DBO). Note that in SQL Server 2000, you need 

to specify the table’s owner rather than the schema 
name as the second element in the fully qualified 
table name. A table’s owner has full rights to alter 
the table’s structure and contents, including the right 
to delete it. In SQL Server 2005, Microsoft opted to 
separate users and schemas, so the second element 
in the fully qualified table name became the schema 
name. The reason for this change is beyond the 
scope of this introductory lesson. If you'd like to 
learn why, read “User-Schema Separation” in SQL 
Server 2005 BOL at msdn2.microsoft.com/en-us/ 
library/ms190387.aspx. 

* TableName is the name of the table you want to 
create. Table names can be a maximum length of 
128 characters and must follow the rules of identi- 
fiers. If you’re unfamiliar with identifiers, see SQL 
Server 2005 BOL’s documentation at msdn2.micro 


soft.com/en-us/library/ms175874.aspx. 


Inside the parentheses is where you list your column 
definitions, each of which must include a column 
name and column data 
type. Column names are 
limited to 128 characters, 
must be unique within the 
table, and must follow the 
identifier rules. Column 
data types must be valid 
SQL Server data types 
(e.g., integer, varchar, 
char). If you're unfa- 
miliar with SQL Server's 
data types, you can read 
about them in 
SQL Server 2005 
BOL at msdn 
-microsoft.com/ 


en-us/library/ 


ms187752.aspx. 
When defining 


columns, you can 
also specify con- 
straints, whether 
null values are 
allowed, and 
other optional 
arguments (more 
on this shortly). 
Now that 
all the technical 
information is 
out of the way, 
it’s time to start 


mr.MovieID 
m.GenreID 


having fun creating your own tables. The following 
examples assume that the current database is MyDB 
and you're the DBO, so the optional DatabaseName 
and SchemaName arguments aren’t included. When 
you run the script, SQL Server will create the table, 
which will be owned by the dbo schema, in MyDB. The 
MyDB database needs to include the Employee table 
from Lesson 3 and the Movie and Genre tables from 
Lesson 5. 

Some of the examples provided in Lesson 4 used 
the MovieReview table, which contained four columns: 
EmployeeID (employee IDs), Genre (textual descrip- 
tions for movies’ genres), MovieName (textual descrip- 
tions for movies’ names), and Stars (movies’ ratings, 
where | star is the worst rating and 5 stars is the best 
rating). Let’s create a new version of the MovieReview 
table that contains IDs rather than textual descriptions 
for the movie names and that doesn’t include a Genre 
column. (The genre information will be obtained later 
through a join operation.) 

Listing 1 shows the code to create the new and 
improved MovieReview table. This code begins by 
deleting the original table if it exists. To do so, the 
code executes the OBJECT_ID function, passing 
in the name of the target table (MovieReview) as a 
parameter. If the table exists in the current database, 
the function returns the table’s object ID. If the table 
doesn’t exist, the function returns a null value. If the 
value isn’t null, the DROP TABLE command deletes 
the MovieReview table. If the value is null, the DROP 
TABLE command is skipped. (Trying to use DROP 
TABLE to delete a non-existent table raises an error.) 

Next, the code uses the CREATE TABLE com- 
mand to create the new MovieReview table. The table 
has four columns: MovieReviewID, EmployeeID, 
MovieID, and Stars. 

The MovieReviewID column is an identity column, 
which is specified by the IDENTITY argument. As I 
explained in Lesson 2, the identity column is special in 
that the data for this column is automatically added 
and its values incremented with each record being 
inserted. The (1,1) parameter indicates that the values 
in the IDENTITY column should start at number 1 
and increment by 1 each time a record is inserted into 
the table. The PRIMARY KEY argument indicates 
that this column will be used to uniquely identify 
records in this table. 

The EmployeeID and MovieID columns contain 
the employee and movie IDs, respectively. The Stars 
column contains a number representing how many 
stars the movie received. In these three columns, notice 
the use of NOT NULL in the column definitions. 
NULL is the term applied to a missing or unknown 
value. By designating a column as being NOT NULL, 
you're telling SQL Server that every record in the 
table must have a known value in that column. So, 
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if you tried to insert a record that 
had NULL for EmployeeID, SQL 
Server would reject the record and 
raise an error. In other words, the 
table will accept only valid integers. 
However, this doesn’t mean that 
the table will accept only correct 
values. If you want to allow only 
correct values to be entered in a 
column, you need to create con- 
straints. Constraints are beyond the 
scope of this article, but if you'd 
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like more information about them 

see “Enforcing Data Integrity” in 

SQL Server 2005 BOL at msdn2 

-microsoft.com/en-us/library/ms190765.aspx. 

NOT NULL, IDENTITY, (1,1), and PRIMARY 
KEY are only a few of the many optional argu- 
ments you can include in column definitions. See the 
CREATE TABLE documentation in SQL Server 2005 
BOL for a complete list. 

You can find the code in Listing 1 in the CodeTo- 
CreateNewMovieReviewTable.sq] file, which is part of 
the 99448 zip file. Run this code in the query window 
in SQL Server 2005’s SQL Server Management Studio 
(SSMS) or SQL Server 2000’s Query Analyzer. 

After you create the new MovieReview table, you 
need to populate it. Run the code in the CodeTo 
PopulateNewMovieReviewTable.sq] file, which is also 
part of the 99448 zip file, in the query window. 

Now let’s use the new MovieReview table to get a list 
of who reviewed which movies, the reviewers’ ratings, and 
the movies’ genres. Listing 2 shows the code to accom- 
plish this. This code performs three inner joins: 

e The MovieReview table is joined with the Employee 
table to retrieve the first and last names of the 
employees who reviewed the movies. The first names 
and last names come from the Employee table’s 
FirstName and LastName columns, respectively. 
The first and last names are concatenated together 
with a space in between. The column containing 
this information is titled Reviewer. 

e The MovieReview table is joined with the Movie 
table to get the movie names from the Movie Table’s 
Movie column. 

e The MovieReview table is joined with the Genre 
table to get the genre name from the Genre table’s 
Genre column. 


The Stars column of the new MovieReview table pro- 
vides the movie ratings. However, instead of displaying 
each rating as a number, the REPLICATE function dis- 
plays an asterisk (*) for every star a movie received. (For 
a detailed explanation of how this works, see Lesson 4.) 

Only movies that received more than 3 stars are 
included in the results, which are ordered first by 
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descending number of stars, then alphabetically by 
genre, and finally alphabetically by movie name. 

You can find the code in Listing 2 in the CodeToList 
WhoReviewedWhichMovies.sql file, which is part 
of 99448.zip. After you execute this code in the query 
window, you should get results like that in Figure 1. 


Using the SELECT INTO 
Command 
The second way to create a table is to use the SELECT 
INTO command to query a database and store the 
results in a new table. This technique is fairly straight- 
forward. Plus, it lets you create and populate the target 
table with one command. However, it lacks the flex- 
ibility offered by the CREATE TABLE command. For 
example, when you use the SELECT INTO command 
to create a table, the table’s columns will be of the 
same datatype as the columns in the SELECT state- 
ment. When you use the CREATE TABLE statement, 
you have the option of customizing each column’s 
datatype. 

The basic syntax to follow when you're using the 
SELECT INTO command to create a table is 


SELECT Columni, 
Column2, 
Column3 
FROM YourExistingTable 
INTO NewTable 


where YourExistingTable is the name of the existing 
table (or tables) being queried and NewTable is the 
name of the table you're creating. (The new table’s 
name can’t already exist in the database.) In this case, 
the SELECT statement selects three columns from the 
existing table for inclusion in the new table, but you 
can use any valid SELECT statement to create the 
table. SQL Server will automatically assign appropriate 
column definitions based on the expressions in the 
SELECT statement. 

Listing 3 shows this technique in action. This code 
joins the MovieReview table with the Movie table to 


Figure | 
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the reviewers’ ratings, 
and the movies’ genres 
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Figure 2 


Result set showing a 
summary of the movie Defragger Hill 
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get the movie names and the MovieReview table 
with the Genre table to get the genre names. The 
Stars column in the MovieReview table provides 
the movie ratings, which are averaged for each 
movie. The average ratings are displayed as 
asterisks rather than numbers. The Count(*) 
function is used to count how many reviews 
each movie has; these counts are assigned to 
a column named Reviews. After grouping the 
results by movie name, then genre, the code 
stores the result set in the MovieReviewSum- 
mary table. 

You can find the code in Listing 3 in the CodeTo 
Create&PopulateMovieReviewSummaryTable 
«sql file, which is part of 99448 zip. After you 
execute this code in the query window, you can 
run the query 


SELECT * 
FROM MovieReviewSummary 
ORDER BY Stars, Reviews 


to get the result set in Figure 2. 

If you were to look at the structure of the new 
table to see how SQL Server defines the columns, 
you'd see that the Movie and Genre columns 
are the same type and length in the new table as 
they were in the source tables. Because the Stars 
and Reviews columns were computed within the 
SELECT statement and not directly pulled from 
physical tables, their type and length matches the 
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result of the corresponding expressions. 


Knowing How to Create 
Tables Isn’t Knowledge 
Reserved Just for DBAs 
Creating tables doesn’t have to be the exclusive 
domain of DBAs. Application developers, 
report writers, business analysts, or anyone else 
who stores data in a relational database will 
benefit from knowing how to create their own 
tables. Being able to write the table creation 
scripts when requesting new tables goes a long 
way in convincing the DBA to do your bidding. 
SOU 
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S Server 2008°s new statements extend 
Q the T-SQL feature set. Let’s take a 
guided tour of some of the most important state- 
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more T-SQL enhancements you'll find useful. 


DECLARE Statement 

With DECLARE, SQL Server 2008 offers the ability 
to assign values to a variable in its declaration. You 
can assign values to most data types, including 
SQLCLR data types, but not to TEXT, NTEXT, 
or IMAGE data types. Here’s an example: 


DECLARE @MyName varchar(2@)=’Michael’ 


Compound Assignment Operators 
SQL Server 2008 supports compound assignments 
and provides these compound operators: +=, -=, 
/=, %=, &=, |=, and ^=. The following statement 
uses the new compound addition operator to add 
two plus two: 


DECLARE @MyNumber int = 2 
SET @MyNumber += @myNumber 


Row Constructor 

Sometimes called Table Value Constructors, SQL 
Server 2008’s new Row Constructor lets you insert 
multiple rows by using a single statement. In the fol- 
lowing example, I used a single INSERT statement 
to insert three rows in a table named MyTable: 


“New Data Types in 


T-SQL Enhancements in SQL Server 2008 


CREATE TABLE MyTable 
CID int, Item varchar (2@)) 
INSERT INTO MyTable VALUES 
(1, ‘Bike’), (2, ‘Car’), (3, ‘Truck’) 


MERGE Statement 

You can selectively merge the contents of two tables. To 
merge the previous example, MyTable, with the contents of 
a new table, MyTable2, which contains duplicate data as well 
as new data, I used the MERGE statement 


CREATE TABLE MyTable2 

CID int, Item varchar(2@)) 
INSERT INTO MyTable2 VALUES 

(1, ‘Bike’), (2, ‘Van’), (4, ‘Motorcycle’) 
GO 
MERGE MyTable2 AS TargetTable 
USING (SELECT ID, Item FROM MyTable) 
SourceTable 
ON (TargetTable.ID = SourceTable. 1D) 
WHEN TARGET NOT MATCHED 

THEN INSERT VALUES (ID, Item) 
WHEN SOURCE NOT MATCHED 

THEN UPDATE SET Item = Item; 
GO 


where MyTable2 is the merge target (i.e., the table to be 
updated with merged data) and My Table is the merge source. 
If the value in the ID column is matched, the Item value is 
updated. Otherwise a new row is inserted in MyTable2. Eel 
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